problem with `os.stat` within `eyed3.load` when running multiple tests using `fs` based fixture #1105

stevenengland · 2024-12-26T12:01:38Z

Describe the bug
Hello together, I have hard times finding out what causes the following error: I am using the fs (not fs_module etc. -> fs) inside a fixture. There is one test, that, when ran in isolation, succeeds. But when it runs together (sequentially) with any other test that uses the fs based fixture it fails.

TLDR
Of course the first thought is, are there any other filesystem actions colliding with the test but I think i ruled those out:

1: fs is used on a per test basis (as far as I can read from the docs only fs_module etc. are scoped broader than function, right?)

2: I added some debugging lines that verify the existence of the file which a code line later causes a fail. Also, before the failing line os.stat is called within eyed3.load method, eyed3 performs some internal checks like if not path.is_file() that are passed. So the pure existance of the file should be given I think. more can be read here: https://github.com/nicfit/eyeD3/blob/2cdef0d7542e65cc2756d63c2afffbdf48061eea/eyed3/mp3/__init__.py#L93

How To Reproduce
This is the test setup and test code up to the point where it raises:

# conftest.py
@pytest.fixture(scope="function")
def test_fs(fs):
    yield fs

# testmethod
    @e2e
    def test_copy_audio_with_fs(
        self,
        sut: AudioFileManager,
        test_assets_dir: str,
        test_fs,
    ):
        # GIVEN
        test_fs.add_real_directory(str(test_assets_dir))
        source_file_path = os.path.join(test_assets_dir, "0001.mp3")
        target_file_path = os.path.join(
            test_assets_dir,
            "target_repo",
            "9999.mp3",
        )
        os.makedirs(
            os.path.join(test_assets_dir, "target_repo"),
        )
        # WHEN
        sut.copy_audio( # <-- this line calls, besides others, the function _read_audio() that raises
            source_file_path,
            target_file_path,
        )

The following code is where it leaves my codebase and raises the error within eyed3:

    def _read_audio(self, file_path: str) -> eyed3.AudioFile:
        # some debugging lines ...
        print(f"file_path: {file_path}")
        print(f"files in parent dir: {os.listdir(os.path.dirname(file_path))}")
        print(f"is parent a dir: {os.path.isdir(os.path.dirname(file_path))}")
        print(f"is file_path a file: {os.path.isfile(file_path)}")
        print(f"size_bytes: {os.stat(file_path)[stat.ST_SIZE]}") # <-- this is the line used in eyed3.load internally that throws, see down below the stack trace
        audio = eyed3.load(file_path) # <-- this line throws when reaching the code line equal to the above one, see down below the stack trace
        # more stuff here that is irrelevant, because the line before fails
        return audio

The resulting debugging output is this:

file_path: [...]\tests\test_assets\target_repo\9999.mp3
files in parent dir: ['9999.mp3']
is parent a dir: True
is file_path a file: True
size_bytes: 3650

The error thrown/stacktrace is:

    sut.copy_audio(
[...]\src\mp3\audio_file_manager.py:52: in copy_audio
    audio = self._read_audio(target_path)
[...]\src\mp3\audio_file_manager.py:81: in _read_audio
    audio = eyed3.load(file_path)
[...]\.venv\Lib\site-packages\eyed3\core.py:452: in load
    return mp3.Mp3AudioFile(path, tag_version)
[...]\.venv\Lib\site-packages\eyed3\mp3\__init__.py:146: in __init__
    super().__init__(path)
[...]\.venv\Lib\site-packages\eyed3\core.py:239: in __init__
    self._read()
[...]\.venv\Lib\site-packages\eyed3\mp3\__init__.py:164: in _read
    self._info = Mp3AudioInfo(file_obj, mp3_offset, self._tag)
[...]\.venv\Lib\site-packages\eyed3\mp3\__init__.py:93: in __init__
    size_bytes = os.stat(file_obj.name)[stat.ST_SIZE]
[...]\.venv\Lib\site-packages\pyfakefs\fake_os.py:1426: in wrapped
    return f(*args, **kwargs)
[...]\.venv\Lib\site-packages\pyfakefs\fake_os.py:693: in stat
    return self.filesystem.stat(path, follow_symlinks)
[...]\.venv\Lib\site-packages\pyfakefs\fake_filesystem.py:673: in stat
    file_object = self.resolve(
[...]\.venv\Lib\site-packages\pyfakefs\fake_filesystem.py:1760: in resolve
    return self.get_object_from_normpath(
[...]\.venv\Lib\site-packages\pyfakefs\fake_filesystem.py:1689: in get_object_from_normpath
    self.raise_os_error(errno.ENOENT, path)
[...]\.venv\Lib\site-packages\pyfakefs\fake_filesystem.py:430: in raise_os_error
    raise OSError(err_no, message, filename)
E   FileNotFoundError: [Errno 2] No such file or directory in the fake filesystem: '[...]\\tests\\test_assets\\target_repo\\9999.mp3'

Your environment
Windows-11-10.0.22000-SP0
Python 3.12.3 (tags/v3.12.3:f6650f9, Apr 9 2024, 14:05:25) [MSC v.1938 64 bit (AMD64)]
pyfakefs 5.6.0
pytest 8.2.1

does such a scenario ring a bell of someone? What can I do to further track down the issue?

Kind regards!

The text was updated successfully, but these errors were encountered:

mrbean-bremen · 2024-12-26T16:43:23Z

Thanks for the report!
I cannot say out of the box what the problem is - it may be a problem with the package being incompatible with pyfakefs, a bug in pyfakefs or something else.
I will try to reproduce it given your information, though a self-contained reproducible example would be helpful, of course.

mrbean-bremen · 2024-12-26T16:59:35Z

If using your code with a dummy AudioFileManager that in copy_audio just calls _read_audio, I cannot reproduce the problem. Note that I have omitted the e2e decorator - I don't know it and how it works with pytest. I also did not see anything in the eyed3.load function that looks like it could not be patched, but I may have missed something.

I would need a bit more information to be able to reproduce this, ideally a self-contained example, as I wrote above.

stevenengland · 2024-12-26T18:37:36Z

Hi, I'll try to provide a minimal example tomorrow.

stevenengland · 2024-12-27T08:54:22Z

Hi @mrbean-bremen , here is a minimal example that raises for me:

@pytest.fixture(scope="function")
def test_fs(fs):
    yield fs

class TestFailingE2ETests:

    def test_read_audio_content_and_id3_tags(
        self,
        test_fs: FakeFilesystem,
    ):
        test_assets_dir: str = (
            "[...]\\tests\\test_assets"
        )

        test_fs.add_real_directory(test_assets_dir)
        audio = eyed3.load(os.path.join(test_assets_dir, "0002.mp3"))
        assert audio is not None

    def test_copy_audio_with_fs(
        self,
        test_fs: FakeFilesystem,
    ):
        # GIVEN
        test_assets_dir: str = (
            "[...]\\tests\\test_assets"
        )

        test_fs.add_real_directory(test_assets_dir)
        source_file_path = os.path.join(test_assets_dir, "0001.mp3")
        target_file_path = os.path.join(
            test_assets_dir,
            "target_repo",
            "9999.mp3",
        )
        test_fs.create_dir(
            os.path.join(test_assets_dir, "target_repo"),
        )

        shutil.copyfile(source_file_path, target_file_path)

        audio = eyed3.load(target_file_path)

        assert audio is not None

Each test running in isolation succeeds. But executing both tests in a sequence raises:

tests\test_audio_file_manager.py .F                                      [100%] <-- first test passes, second does not

================================== FAILURES ===================================
_________________ TestFailingE2ETests.test_copy_audio_with_fs _________________
[...]\.venv\Lib\site-packages\pyfakefs\fake_filesystem.py:1750: in get_object_from_normpath
    target = target.get_entry(component)  # type: ignore
[...]\.venv\Lib\site-packages\pyfakefs\fake_file.py:566: in get_entry
    return self.entries[to_string(pathname_name)]
E   KeyError: 'target_repo'

During handling of the above exception, another exception occurred:
[...]\tests\test_audio_file_manager.py:235: in test_copy_audio_with_fs
    audio = eyed3.load(target_file_path)
[...]\.venv\Lib\site-packages\eyed3\core.py:452: in load
    return mp3.Mp3AudioFile(path, tag_version)
[...]\.venv\Lib\site-packages\eyed3\mp3\__init__.py:146: in __init__
    super().__init__(path)
[...]\.venv\Lib\site-packages\eyed3\core.py:239: in __init__
    self._read()
[...]\.venv\Lib\site-packages\eyed3\mp3\__init__.py:164: in _read
    self._info = Mp3AudioInfo(file_obj, mp3_offset, self._tag)
[...]\.venv\Lib\site-packages\eyed3\mp3\__init__.py:93: in __init__
    size_bytes = os.stat(file_obj.name)[stat.ST_SIZE]
[...]\.venv\Lib\site-packages\pyfakefs\fake_os.py:1456: in wrapped
    return f(*args, **kwargs)
[...]\.venv\Lib\site-packages\pyfakefs\fake_os.py:694: in stat
    return self.filesystem.stat(path, follow_symlinks)
[...]\.venv\Lib\site-packages\pyfakefs\fake_filesystem.py:732: in stat
    file_object = self.resolve(
[...]\.venv\Lib\site-packages\pyfakefs\fake_filesystem.py:1834: in resolve
    return self.get_object_from_normpath(
[...]\.venv\Lib\site-packages\pyfakefs\fake_filesystem.py:1761: in get_object_from_normpath
    self.raise_os_error(errno.ENOENT, path)
[...]\.venv\Lib\site-packages\pyfakefs\fake_filesystem.py:489: in raise_os_error
    raise OSError(err_no, message, filename)
E   FileNotFoundError: [Errno 2] No such file or directory in the fake filesystem: '[...]\\tests\\test_assets\\target_repo\\9999.mp3'
=========================== short test summary info ===========================
FAILED tests/test_audio_file_manager.py::TestFailingE2ETests::test_copy_audio_with_fs
========================= 1 failed, 1 passed in 0.66s =========================
Finished running tests!

As you can see it complains E KeyError: 'target_repo'. But I don't have a clue why.

mrbean-bremen · 2024-12-27T18:17:35Z

Thanks a lot, I now can reproduce the problem! Still a bit at a loss as to the cause, but I will see what I can find.

stevenengland · 2024-12-27T19:48:13Z

Thanks a lot in advance! And good luck :-)

mrbean-bremen · 2024-12-30T20:12:13Z

I'm quite sure that this is a bug in pyfakefs regarding the patching of pathlib by the dynamic patcher, but it may take some time to understand and fix it. For the time being you could try to switch off the dynamic patcher. You can use:

@pytest.fixture
def test_fs():
    with Patcher(use_dynamic_patch=False) as patcher:
        yield patcher.fs

as your fixture. This will fix the minimal test, but the dynamic patcher may still be needed for other tests, so this may or may not help you.

mrbean-bremen · 2025-01-13T20:57:25Z

I kind of got stuck with this issue, as I could not find an automatic fix for the problem, and I'm not sure if I will find one - I currently consider this a limitation of pyfakefs.
There is a workaround though that will fix this kind of issue by reloading the problematic module in your fixture:

def reload_cleanup_handler(name):
    if name in sys.modules:
        importlib.reload(sys.modules[name])
    return True

@pytest.fixture
def test_fs():
    with Patcher() as patcher:
        patcher.cleanup_handlers["eyed3.mimetype"] = reload_cleanup_handler
        yield patcher.fs

I've also made a PR (now merged) that adds this specific reload_cleanup_handler as a convenience function and fixes the documentation for the cleanup handlers in the troubleshouting chapter.

Note that in an earlier version I had done this reload to all dynamically loaded modules, but that turned out not to be a good idea because of certain unpredictable side effects in some modules.
I will think about this some more - maybe I'll find a way to detect which modules need to be reloaded, but for now, setting the cleanup handler is the way to go.

stevenengland · 2025-01-14T17:55:36Z

Thank you very much for your efforts. I'll give your convenience function a try this week.

Thanks

mrbean-bremen · 2025-01-14T19:27:19Z

FYI: I just made a patch release which contains these changes (among a couple of other fixes).

stevenengland · 2025-01-19T14:22:02Z

Hi @mrbean-bremen , I just updated pyfakefs to the latest patched version but ... unfortunately I am not quite sure what I need to implement to make it work. I think I need to follow your advice for dynamic patcher with patching a module (https://pytest-pyfakefs.readthedocs.io/en/latest/troubleshooting.html#tests-failing-after-a-test-using-pyfakefs). Returning to the minimal example with eyed3: Is the module my module that uses eyed3 or is it eyed3 or something within eyed3?

Thanks!

mrbean-bremen · 2025-01-19T14:49:21Z

It is actually eyed3.mimetype, that's what I put in the fixture code above, e.g. use this fixture instead of fs:

from pyfakefs.helpers import reload_cleanup_handler

@pytest.fixture
def test_fs():
    with Patcher() as patcher:
        patcher.cleanup_handlers["eyed3.mimetype"] = reload_cleanup_handler
        yield patcher.fs

mrbean-bremen · 2025-01-19T14:56:10Z

I would still like to have that done automatically, but I haven't figured out a way to do this yet.

stevenengland · 2025-01-19T15:21:10Z

Thanks though for the manual way. But I still do not get it up and running.

Maybe in my case I need some more patching... Since these tests pass in single mode I will maybe wait for an automatic solution.

For my own reference: https://github.com/stevenengland/dfplayer_card_manager/blob/89ce0d9c6497b4755a28ed3e15066bbef7b23a51/tests/conftest.py#L20

mrbean-bremen · 2025-01-19T16:00:36Z

Ok, I will have a look at your project a bit later.

mrbean-bremen · 2025-01-19T17:23:20Z

Can you point to the tests that fail in your project and pass if run in single mode? I'm trying to reproduce the problem in your repo.

mrbean-bremen added the bug label Dec 30, 2024

mrbean-bremen mentioned this issue Jan 13, 2025

Fix cleanup handler documentation #1111

Merged

6 tasks

mrbean-bremen added the limitation label Jan 13, 2025

mrbean-bremen removed the bug label Jan 14, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

problem with `os.stat` within `eyed3.load` when running multiple tests using `fs` based fixture #1105

problem with `os.stat` within `eyed3.load` when running multiple tests using `fs` based fixture #1105

stevenengland commented Dec 26, 2024

mrbean-bremen commented Dec 26, 2024

mrbean-bremen commented Dec 26, 2024

stevenengland commented Dec 26, 2024

stevenengland commented Dec 27, 2024 •

edited

Loading

mrbean-bremen commented Dec 27, 2024

stevenengland commented Dec 27, 2024

mrbean-bremen commented Dec 30, 2024

mrbean-bremen commented Jan 13, 2025 •

edited

Loading

stevenengland commented Jan 14, 2025

mrbean-bremen commented Jan 14, 2025

stevenengland commented Jan 19, 2025

mrbean-bremen commented Jan 19, 2025

mrbean-bremen commented Jan 19, 2025

stevenengland commented Jan 19, 2025

mrbean-bremen commented Jan 19, 2025

mrbean-bremen commented Jan 19, 2025

problem with os.stat within eyed3.load when running multiple tests using fs based fixture #1105

problem with os.stat within eyed3.load when running multiple tests using fs based fixture #1105

Comments

stevenengland commented Dec 26, 2024

mrbean-bremen commented Dec 26, 2024

mrbean-bremen commented Dec 26, 2024

stevenengland commented Dec 26, 2024

stevenengland commented Dec 27, 2024 • edited Loading

mrbean-bremen commented Dec 27, 2024

stevenengland commented Dec 27, 2024

mrbean-bremen commented Dec 30, 2024

mrbean-bremen commented Jan 13, 2025 • edited Loading

stevenengland commented Jan 14, 2025

mrbean-bremen commented Jan 14, 2025

stevenengland commented Jan 19, 2025

mrbean-bremen commented Jan 19, 2025

mrbean-bremen commented Jan 19, 2025

stevenengland commented Jan 19, 2025

mrbean-bremen commented Jan 19, 2025

mrbean-bremen commented Jan 19, 2025

problem with `os.stat` within `eyed3.load` when running multiple tests using `fs` based fixture #1105

problem with `os.stat` within `eyed3.load` when running multiple tests using `fs` based fixture #1105

stevenengland commented Dec 27, 2024 •

edited

Loading

mrbean-bremen commented Jan 13, 2025 •

edited

Loading