Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bad assertion with pytest 7.0.0 & fs fixture #666

Closed
Wescoeur opened this issue Feb 9, 2022 · 6 comments · Fixed by #685
Closed

Bad assertion with pytest 7.0.0 & fs fixture #666

Wescoeur opened this issue Feb 9, 2022 · 6 comments · Fixed by #685
Labels

Comments

@Wescoeur
Copy link

Wescoeur commented Feb 9, 2022

Describe the bug
With the new pytest version (7.0.0), we can't run pytest/pyfakes using this command line with a subfolder:

pytest tests/

Instead we must use:

cd tests/
pytest

Otherwise we have this trace:

> pytest tests/
/home/ronan/.local/lib/python3.10/site-packages/_pytest/nodes.py:664: PytestWarning: PylamaItem is an Item subclass and should not be a collector, however its bases File are collectors.
Please split the Collectors and the Item into separate node types.
Pytest Doc example: https://docs.pytest.org/en/latest/example/nonpython.html
example pull request on a plugin: https://github.com/asmeurer/pytest-flakes/pull/40/
  warnings.warn(
=========================================================================================================== test session starts ============================================================================================================
platform linux -- Python 3.10.1, pytest-7.0.0, pluggy-0.13.1
rootdir: /home/ronan/Projets/pyfakefs-pytest-bug/tests, configfile: pytest.ini
plugins: pyfakefs-4.5.4, pylama-7.7.1
collected 2 items                                                                                                                                                                                                                          

tests/test_issue.py::TestIssue::test_ok PASSED                                                                                                                                                                                       [ 50%]
tests/test_issue.py::TestIssue::test_bad Traceback (most recent call last):
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/main.py", line 268, in wrap_session
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/main.py", line 322, in _main
  File "/usr/lib/python3.10/site-packages/pluggy/hooks.py", line 286, in __call__
  File "/usr/lib/python3.10/site-packages/pluggy/manager.py", line 93, in _hookexec
  File "/usr/lib/python3.10/site-packages/pluggy/manager.py", line 84, in <lambda>
  File "/usr/lib/python3.10/site-packages/pluggy/callers.py", line 208, in _multicall
  File "/usr/lib/python3.10/site-packages/pluggy/callers.py", line 80, in get_result
  File "/usr/lib/python3.10/site-packages/pluggy/callers.py", line 187, in _multicall
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/main.py", line 347, in pytest_runtestloop
  File "/usr/lib/python3.10/site-packages/pluggy/hooks.py", line 286, in __call__
  File "/usr/lib/python3.10/site-packages/pluggy/manager.py", line 93, in _hookexec
  File "/usr/lib/python3.10/site-packages/pluggy/manager.py", line 84, in <lambda>
  File "/usr/lib/python3.10/site-packages/pluggy/callers.py", line 208, in _multicall
  File "/usr/lib/python3.10/site-packages/pluggy/callers.py", line 80, in get_result
  File "/usr/lib/python3.10/site-packages/pluggy/callers.py", line 187, in _multicall
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/runner.py", line 113, in pytest_runtest_protocol
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/runner.py", line 132, in runtestprotocol
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/runner.py", line 225, in call_and_report
  File "/usr/lib/python3.10/site-packages/pluggy/hooks.py", line 286, in __call__
  File "/usr/lib/python3.10/site-packages/pluggy/manager.py", line 93, in _hookexec
  File "/usr/lib/python3.10/site-packages/pluggy/manager.py", line 84, in <lambda>
  File "/usr/lib/python3.10/site-packages/pluggy/callers.py", line 208, in _multicall
  File "/usr/lib/python3.10/site-packages/pluggy/callers.py", line 80, in get_result
  File "/usr/lib/python3.10/site-packages/pluggy/callers.py", line 187, in _multicall
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/terminal.py", line 541, in pytest_runtest_logreport
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/terminal.py", line 862, in _locationline
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/terminal.py", line 852, in mkrel
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1057, in cwd_relative_nodeid
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/pathlib.py", line 688, in bestrelpath
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/main.py", line 288, in wrap_session
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1047, in notify_exception
  File "/usr/lib/python3.10/site-packages/pluggy/hooks.py", line 286, in __call__
  File "/usr/lib/python3.10/site-packages/pluggy/manager.py", line 93, in _hookexec
  File "/usr/lib/python3.10/site-packages/pluggy/manager.py", line 84, in <lambda>
  File "/usr/lib/python3.10/site-packages/pluggy/callers.py", line 208, in _multicall
  File "/usr/lib/python3.10/site-packages/pluggy/callers.py", line 80, in get_result
  File "/usr/lib/python3.10/site-packages/pluggy/callers.py", line 187, in _multicall
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/capture.py", line 799, in pytest_internalerror
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/capture.py", line 667, in stop_global_capturing
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/capture.py", line 585, in stop_capturing
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/capture.py", line 412, in done
  File "/home/ronan/.local/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py", line 3882, in close
  File "/home/ronan/.local/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py", line 1423, in get_open_file
  File "/home/ronan/.local/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py", line 1058, in raise_os_error
OSError: [Errno 9] Bad file descriptor in the fake filesystem: '5'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ronan/.local/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py", line 2070, in get_object_from_normpath
  File "/home/ronan/.local/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py", line 729, in get_entry
KeyError: 'home'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/config/__init__.py", line 165, in main
  File "/usr/lib/python3.10/site-packages/pluggy/hooks.py", line 286, in __call__
  File "/usr/lib/python3.10/site-packages/pluggy/manager.py", line 93, in _hookexec
  File "/usr/lib/python3.10/site-packages/pluggy/manager.py", line 84, in <lambda>
  File "/usr/lib/python3.10/site-packages/pluggy/callers.py", line 208, in _multicall
  File "/usr/lib/python3.10/site-packages/pluggy/callers.py", line 80, in get_result
  File "/usr/lib/python3.10/site-packages/pluggy/callers.py", line 187, in _multicall
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/main.py", line 315, in pytest_cmdline_main
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/main.py", line 300, in wrap_session
  File "/home/ronan/.local/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py", line 4010, in chdir
  File "/home/ronan/.local/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py", line 3144, in confirmdir
  File "/home/ronan/.local/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py", line 2125, in resolve
  File "/home/ronan/.local/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py", line 2075, in get_object_from_normpath
  File "/home/ronan/.local/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py", line 1058, in raise_os_error
FileNotFoundError: [Errno 2] No such file or directory in the fake filesystem: '/home/ronan/Projets/pyfakefs-pytest-bug'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ronan/.local/bin/pytest", line 8, in <module>
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/config/__init__.py", line 188, in console_main
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/config/__init__.py", line 173, in main
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1001, in _ensure_unconfigure
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/capture.py", line 667, in stop_global_capturing
  File "/home/ronan/.local/lib/python3.10/site-packages/_pytest/capture.py", line 582, in stop_capturing
ValueError: was already stopped

How To Reproduce
A minimal code is present here to reproduce the bug: https://github.com/Wescoeur/pyfakefs-pytest-bug
Use directly pytest tests/ in the root folder of this repo.
Note: If log_cli is disabled in pytest.ini, the assertion of the stacktrace is always true and we don't have the issue. The issue is triggered when log_cli is true and when the fs fixture is used in a test like in:

    def test_bad(self, fs):
        pass

Your environment
Linux-5.16.1-arch1-1-x86_64-with-glibc2.33
Python 3.10.1 (main, Dec 18 2021, 23:53:45) [GCC 11.1.0]
pyfakefs 4.5.4

@mrbean-bremen
Copy link
Member

Thanks for the report - this is weird! From the first glance, it looks like pytest itself uses some file access while pyfakefs is still active, e.g. fs access is patched.
I can reproduce this with your code example (and confirm that it works with the previous pytest version), and I will try to find the cause. Right now I have no idea how easy or complicated a fix is, so this may take a while.

@mrbean-bremen
Copy link
Member

mrbean-bremen commented Feb 9, 2022

Ok, for my own reference:
The crash happens due to new assert statements in _pytest.pathlib:

def bestrelpath(directory: Path, dest: Path) -> str:
    """Return a string which is a relative path from directory to dest such
    that directory/bestrelpath == dest.

    The paths must be either both absolute or both relative.

    If no such path can be determined, returns dest.
    """
    assert isinstance(directory, Path)  # new in pytest 7
    assert isinstance(dest, Path)
    ...   # unchanged code

If commenting out these asserts in pytest, the test passes. The asserts have been added for this issue to make sure pathlib Paths are used and not strings.
In our case, directory and dest are of the correct type (pathlib.WindowsPath or pathlib.PosixPath), but Path in the pytest module is still mocked, so the isinstance check fails.

As for the fix, I don't have an idea yet - the clean fix would be to make sure that nothing is patched at that point, but I'm not sure how to achieve this.

@Wescoeur
Copy link
Author

Thanks for your feedback! :)

Right now I have no idea how easy or complicated a fix is, so this may take a while.

No problem. There is a workaround for the moment. 👍

mrbean-bremen added a commit to mrbean-bremen/pyfakefs that referenced this issue Feb 13, 2022
- avoids the use of FakePathLibPath in pathlib
- add _pytest.pathlib to skipped modules,
  fake isinstance check for RealPathlibPathModule
- prevents issue with _pytest.pathlib in pytest 7.0.0,
  which would cause all tests with fs fixture to fail
- see pytest-dev#666
mrbean-bremen added a commit that referenced this issue Feb 13, 2022
- avoids the use of FakePathLibPath in pathlib
- add _pytest.pathlib to skipped modules,
  fake isinstance check for RealPathlibPathModule
- prevents issue with _pytest.pathlib in pytest 7.0.0,
  which would cause all tests with fs fixture to fail
- see #666
github-actions bot pushed a commit that referenced this issue Feb 13, 2022
…ePathLibPath in pathlib - add _pytest.pathlib to skipped modules, fake isinstance check for RealPathlibPathModule - prevents issue with _pytest.pathlib in pytest 7.0.0, which would cause all tests with fs fixture to fail - see #666
@mrbean-bremen
Copy link
Member

@Wescoeur - should be fixed in master now - please check if it works for you!
If you can confirm this, I will make a patch release, as this is a severe issue with pytest 7 now out.

@Wescoeur
Copy link
Author

@mrbean-bremen Hi, I confirm, it's fixed on my side with the last commit! Thx! 👍

@mrbean-bremen
Copy link
Member

mrbean-bremen commented Feb 14, 2022

Thanks - will make a patch release tonight in this case!
EDIT: Done.

mrbean-bremen added a commit to mrbean-bremen/pyfakefs that referenced this issue Jul 10, 2022
- fixes pytest-dev#666 without the need to skip _pytest.pathlib patching
- add module and session scoped fs fixtures
mrbean-bremen added a commit to mrbean-bremen/pyfakefs that referenced this issue Jul 10, 2022
- fixes pytest-dev#666 without the need to skip _pytest.pathlib patching
- add module and session scoped fs fixtures
- fixes pytest-dev#684
mrbean-bremen added a commit that referenced this issue Jul 11, 2022
- fixes #666 without the need to skip _pytest.pathlib patching
- add module and session scoped fs fixtures
- fixes #684
github-actions bot pushed a commit that referenced this issue Jul 11, 2022
…thout the need to skip _pytest.pathlib patching - add module and session scoped fs fixtures - fixes #684
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants