Make FakePathlibModule.Path compare equal to pathlib.Path

[Aran-Fey]

Sometimes it happens that a test case compares a fake path to a real path - usually when you've created a pathlib.Path in pytest.mark.parametrize, like so:

def my_function():
    return pathlib.Path()

@pytest.mark.parametrize('expected_result', [pathlib.Path()])
def test_my_function(fs, expected_result):
    assert my_function() == expected_result

This fails with AssertionError: assert WindowsPath('.') == WindowsPath('.').

This could of course be avoided by passing in a string and converting it to a path inside the test, but in reality there isn't always such a convenient workaround. Here's a more realistic example, where an object with loads of attributes - some of which are Paths - is loaded from disk and compared to the expected result:

import pytest
from pathlib import Path
from my_module import MyClass

@pytest.mark.parametrize('attrs', [
    {'foo': False,
     'origin': Path(),
     'bar': 17,
     'paths': [Path('/'), Path('somewhere')]},
])
def test_serialization(fs, attrs):
    obj = MyClass(**attrs)
    obj.dump('myobj.dump')
    
    loaded_obj = MyClass.load_file('myobj.dump')
    assert vars(loaded_obj) == attrs

As I'm sure you can see, having to convert all of those attributes to Paths inside the test is rather annoying. For this reason it would be very convenient if fake Paths would compare equal to real Paths.

[mrbean-bremen]

Thanks for the report! I see your problem, and I think there is an easy fix for this (overwriting __eq__), as long as the comparison is this way around, e.g. comparing the fake path against a real path. Would that be sufficient for you?

[Aran-Fey]

Hmm, I think that would probably be sufficient, yes. I'm not sure if it's a good idea to have an asymmetrical __eq__ though. Might cause some confusing errors if someone ever puts the fake path on the wrong side of the comparison. Wouldn't it be better to go all the way and monkeypatch pathlib.Path.__eq__?

[mrbean-bremen]

Well, the question is where to do this. The actual problem is that the pytest.parametrize decorator is read before any patching occurs - this is one of the instances that are mentioned in the documentation, where patching won't work. I agree that overwriting __eq__ is not the best way to handle this. The corrrect way would be to ensure that the pathlib used in the parameter is patched - I will think about this...

[Aran-Fey]

For now, I've worked around the problem by adding this monkey patch to my conftest.py:

# monkeypatch Path.__eq__ so that pyfakefs FakePaths compare equal to real pathlib.Paths
from pathlib import Path
from pyfakefs.fake_pathlib import FakePath

# Path somehow gets monkeypatched during testing, so in order to have access
# to the original class we'll simply create an instance of it
PATH = object.__new__(Path)

def path_eq(self, other):
    Path = type(PATH)

    if isinstance(other, (Path, FakePath)):
        return str(self) == str(other)

    return super(Path, self).__eq__(other)

Path.__eq__ = path_eq

[mrbean-bremen]

Yes, this is a valid solution, at least for your case. I have been thinking about adding the same monkeypatching to the pytest plugin, but I'm somewhat reluctant to do this, as generally there could be other non-patched fs calls in a parametrize decorator. I will have another look some time later.

[mrbean-bremen]

Getting back to this, I now think that the correct way to resolve this would be to reload the sut module during the test (for example using modules_to_reload in the test setup), as this is the only way that ensures that the code that is read at load time is correctly patched -- see the documentation for how to do this with pytest.
Actually, the fixture in the documentation is too complicated, it could look like this:

import pytest
from pyfakefs.fake_filesystem_unittest import Patcher
import sut

@pytest.fixture
def fs_reload_sut():
    with Patcher(modules_to_reload=[sut]) as p:
        yield p.fs

[mrbean-bremen]

@jmcgeheeiv - instead of closing the issue, I would like to to convert this (and other issues that may be helpful) into a Q/A discussion issue. If you agree, could you enable discussions in the pyfakefs configuration?

[jmcgeheeiv]

Great idea, @mrbean-bremen. Done.

[brendan-simon-indt]

I'm getting the same error. I've put in the pytest.fixture above (tweaked for my code).

@pytest.fixture
def my_fs() :
    with Patcher( modules_to_reload=[my_module] ) as p :
        yield p.fs

Exception has occurred: AssertionError       (note: full exception trace is shown but execution is paused at: test_defaults)
assert PosixPath('/mnt/data') == PosixPath('/mnt/data')

The types for the two variables are:

<class 'pyfakefs.fake_pathlib.FakePathlibModule.PosixPath'>

<class 'pathlib2.PosixPath'>

Other than the fixture above, is there anything else I need to do?

Actually, I'm not sure why it is pathlib2.Path as the unit tests run on Python 3.7 (but my target system runs Python 2.7) - yes I know I need to move forward and migrate to Python3 ;-)

[mrbean-bremen]

We actually don't support pathlib2 officially anymore, as the real pathlib deviates more and more from the pathlib2 implementation (which is compatible to pathlib in Python 3.4, IIRC). It may well be that the problem you have has to do with that, though I can't say for sure. And yeah, it is time to bury Python 2 :)

[brendan-simon-indt]

I have a "work-around" (or maybe it is the correct way to do things?)

My module was instantiating a default instance of a class at the module scope. It is kind of used as a singleton.

# @attr.s( frozen=True )
@attr.s( slots=True )
class Config_Default( Config_Base ) :
    """A frozen (immutable) version of the `Config_Base` class."""
    pass

#! An instance of the (frozen) default config
CONFIG_DEFAULT = Config_Default()

If I instantiate it in in my pytest function then everything works - presumably as the Path objects become pyfake Path objeccts.

    def test_defaults(
            self,
            mocker,     #! pytest-mock test fixture
            my_fs,      #! my pyfakefs test fixture
            ) :

        CONFIG_DEFAULT = Config_Default()

        config = self.get_config( mocker=mocker, fs=my_fs )

        assert  config.foo_path == CONFIG_DEFAULT.foo_path
        ...

Actually this works without the custom test fixture !!

I thought the custom fixture was supposed to reload everything, but maybe I am misunderstanding how it works ?

[mrbean-bremen]

That's indeed one way to solve this: inside your test function, everything is mocked. The problem occurs if some global state which uses fs functions is instantiated before the test run, e.g. before the fake fs is created. This is also an answer to a common question about patching global state that I have seen on StackOverflow often enough.

[brendan-simon-indt]

Ok. Got it now. Thank you :)

If I get a reference to the instance within the test (e.g. my_module.CONFIG_DEFAULT), rather than at the global level (from my_module import CONFIG_DEFAULT), then all is good (using the customised test fixture to reload my_modules)

    # from my_module import CONFIG_DEFAULT, Config_Default

    def test_defaults(
            self,
            mocker,     #! pytest-mock test fixture
            my_fs,      #! my pyfakefs test fixture
            ) :

        # CONFIG_DEFAULT = Config_Default()
        CONFIG_DEFAULT = my_module.CONFIG_DEFAULT

        config = self.get_config( mocker=mocker, fs=my_fs )

        assert  config.foo_path == CONFIG_DEFAULT.foo_path
        ...