New Python 3.9's is_relative_to(), readlink() and with_stem() do not work if pathlib2 is installed

[JD-Veiga]

Hi,

I have noticed that the new Python 3.9's is_relative_to(), readlink() and with_stem() methods have not been ported to pyfakefs yet.

Just in case. I really know that following the pace of Python's new features has been increasingly difficult.

Thank you.

[mrbean-bremen]

Thanks - actually, readlink support has been added in version 4.4.0, and both is_relative_to and with_stem should work out of the box, as they are PurePath implementations (is_relative_to is also tested).
Are you testing with the latest pyfakefs version, and if yes, can you please show an example that is not working?

[JD-Veiga]

I am running pyfakefs 4.4.0 with Python 3.9.3 (on macos 10.14.6).

If I run:

import unittest
import pathlib

import pyfakefs

from pyfakefs import fake_filesystem_unittest


class ExampleTestCase(fake_filesystem_unittest.TestCase):
    
    def setUp(self):
        self.setUpPyfakefs()

    def test_is_relative_to(self):

        self.assertEqual(pyfakefs.__version__, '4.4.0')

        parent = pathlib.Path('/spam/ham')
        file = pathlib.Path(parent, 'eggs')

        self.assertTrue(file.is_relative_to(parent))


# in real filesystem
parent = pathlib.Path('/spam/ham')
file = pathlib.Path(parent, 'eggs')

print()
print('In real filesystem:', file.is_relative_to(parent))
print()
assert file.is_relative_to(parent)


if __name__ == '__main__':
    unittest.main(verbosity=2)

I get the message:

In real filesystem: True

test_is_relative_to (__main__.ExampleTestCase) ... ERROR

======================================================================
ERROR: test_is_relative_to (__main__.ExampleTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jv/Downloads/testing_python2.py", line 21, in test_is_relative_to
    self.assertTrue(file.is_relative_to(parent))
AttributeError: 'PosixPath' object has no attribute 'is_relative_to'

----------------------------------------------------------------------
Ran 1 test in 0.081s

FAILED (errors=1)

[JD-Veiga]

Test for is_relative_to(), with_stem(), and readlink():

import unittest
import pathlib

import pyfakefs

from pyfakefs import fake_filesystem_unittest


class ExampleTestCase(fake_filesystem_unittest.TestCase):
    
    def setUp(self):
        self.setUpPyfakefs()
        self.assertEqual(pyfakefs.__version__, '4.4.0')

    def test_is_relative_to(self):
        parent = pathlib.Path('/spam/ham')
        file = pathlib.Path(parent, 'eggs')
        self.assertTrue(file.is_relative_to(parent))

    def test_with_stem(self):
        file = pathlib.Path('/spam/ham/eggs')
        self.assertEqual(str(file.with_stem('foo')), '/spam/ham/foo')

    def test_readlink(self):
        file = pathlib.Path('/spam/ham/eggs')
        link_to = pathlib.Path('/link')
        link_to.symlink_to(file)
        self.assertEqual(link_to.readlink(), file)


# in real filesystem
parent = pathlib.Path('/spam/ham')
file = pathlib.Path(parent, 'eggs')

print()
print('is_relative_to() in real filesystem:', file.is_relative_to(parent))
assert file.is_relative_to(parent)

print('with_stem() in real filesystem:', file.with_stem('foo'))
assert str(file.with_stem('foo')) == '/spam/ham/foo'

# readlink not tested in real filesystem because side effects

print()


if __name__ == '__main__':
    unittest.main(verbosity=2)

that prints:

is_relative_to() in real filesystem: True
with_stem() in real filesystem: /spam/ham/foo

test_is_relative_to (__main__.ExampleTestCase) ... ERROR
test_readlink (__main__.ExampleTestCase) ... ERROR
test_with_stem (__main__.ExampleTestCase) ... ERROR

======================================================================
ERROR: test_is_relative_to (__main__.ExampleTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jv/Downloads/testing_python2.py", line 18, in test_is_relative_to
    self.assertTrue(file.is_relative_to(parent))
AttributeError: 'PosixPath' object has no attribute 'is_relative_to'

======================================================================
ERROR: test_readlink (__main__.ExampleTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jv/Downloads/testing_python2.py", line 28, in test_readlink
    self.assertEqual(link_to.readlink(), file)
AttributeError: 'PosixPath' object has no attribute 'readlink'

======================================================================
ERROR: test_with_stem (__main__.ExampleTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jv/Downloads/testing_python2.py", line 22, in test_with_stem
    self.assertEqual(str(file.with_stem('foo')), '/spam/ham/foo')
AttributeError: 'PosixPath' object has no attribute 'with_stem'

----------------------------------------------------------------------
Ran 3 tests in 0.227s

FAILED (errors=3)

I hope that it can help you.

[mrbean-bremen]

Thank you, I will have a look!

[mrbean-bremen]

I could not reproduce this. I added your test (ExampleTestCase) to the CI build, and it passed for all systems (including MacOS). While this is in master, there haven't been any changes since the last release, so I'm not sure what causes the problem for you... The CI build runs with Python 3.9.2, locally I have tested with Python 3.9.1.

[JD-Veiga]

I think that this issue could be related to pathlib2 (as it was when you added support link_to() method -- see: 20ed320 and #580 (comment)).

After uninstalling pathlib2 from my system, tests pass without problem.

Unfortunately, pathlib2 seems to be required by wxPython.

[mrbean-bremen]

I see... I have been thinking about pathlib2, but I thought that this could not be the problem - I was wrong, obviously.
If pathlib2 is used in the usual manner (e.g. import pathlib2 as pathlib), these methods should not work, as pathlib2 does not implement them (pathlib2 is no longer developed after Python2 went EOL), and this is what pyfakefs assumes. If it is installed but not used this might cause the problem.

I'm thinking about removing support for pathlib2 at all, as it was only needed for Python 2.7, which we don't support anymore. I will think about this...
As for wxPython - since version 4.1.0, it uses pathlib2 only for Python 2 - do you use an older version?

[JD-Veiga]

I am using latest version (WxPython 4.1.1).

I imagine that WxPython or pip installs pathlib2 as a dependency by default or as long as you have a Python 2 in your system (and Python 2 is still in my system because some packages still use it --which it is a burden, I must say).

And, as you stated, pyfakefs imports pathlib2 even if it is not used. Can this be solved by checking Python version before importing pathlib2?

Anyway, I have removed pathlib2 completely though I supposed it will come back whenever any update of WxPython (or any other package, who knows) is done.

Perhaps, in long term, the best solution for you is to remove the support for pathlib2 --considering that Python2 is not longer supported by pyfakefs. But this is only my personal opinion. I do not really know if someone still uses it.

Thanks a lot.

[mrbean-bremen]

Thank you! I'm seriously considering to remove pathlib2 support, as I see no real benefit. wxPython uses it only for Python 2, and I would consider using it for a Python version >= 3.6 a bug, as it would remove functionality from pathlib.
Checking the Python version also does not make much sense, as we only support Python versions where pathlib2 is not needed.

[JD-Veiga]

Thank you for your help, @mrbean-bremen. It is always a pleasure speaking with you.

[mrbean-bremen]

@JD-Veiga - I did not remove pathlib2 support completely, but it is not imported as pathlib anymore, so it should work for you now. If you can, please check if current master works for you.

[JD-Veiga]

It works fine. Thanks again.