Test is skipped without a skip mark added to the test

[201dreamers]

• a detailed description of the bug or problem you are having
• output of pip list from the virtual environment you are using
• pytest and operating system versions
• minimal example if possible

Tree of the project:

$ tree -I venv -I __pycache__
.
├── conftest.py
└── test.py

I have 2 test classes that are based on the same class (they use Template Method pattern).
When I add a skip mark to only one test function from the pytest_generate_test hook, both tests are skipped.

test.py contents

class Base:
    def step_1(self):
        pass

    def test_func(self):
        self.step_1()
        assert True


class TestCls1(Base):
    def step_1(self):
        return 1


class TestCls2(Base):
    def step_1(self):
        return 2

conftest.py contents:

import pytest


def pytest_generate_tests(metafunc):
    print(f"\n{metafunc}")
    print(f"{metafunc.function=}")

    if  metafunc.cls.__name__ == "TestCls1":
        metafunc.function.pytestmark = pytest.mark.skip("Skip 1")

Output of the run:

(venv) [15:20:05] check_pytest_issue $ pytest test.py -vv
===================================================================== test session starts =====================================================================
platform linux -- Python 3.12.3, pytest-8.3.4, pluggy-1.5.0 -- /mnt/c/Users/dhakman/check_pytest_issue/venv/bin/python
cachedir: .pytest_cache
rootdir: /mnt/c/Users/dhakman/check_pytest_issue
collecting ...
<_pytest.python.Metafunc object at 0x7fb9d4de7c20>
metafunc.function=<function Base.test_func at 0x7fb9d4e404a0>

<_pytest.python.Metafunc object at 0x7fb9d4df5400>
metafunc.function=<function Base.test_func at 0x7fb9d4e404a0>
collected 2 items

test.py::TestCls1::test_func SKIPPED (Skip 1)                                                                                                           [ 50%]
test.py::TestCls2::test_func SKIPPED (Skip 1)                                                                                                           [100%]

===================================================================== 2 skipped in 0.03s ======================================================================

If you take a look at the output, despite metafunc object is different for each test class, the underlying function object is the same object in both cases (id = 0x7fb9d4e404a0).

If we take a look at plain classes in python, in similar structure they have different ids:

Python 3.12.3 (main, Nov  6 2024, 18:32:19) [GCC 13.2.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.30.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: class Base:
   ...:     def test(self):
   ...:         pass
   ...:

In [2]: class A(Base):
   ...:     pass
   ...:

In [3]: class B(Base):
   ...:     pass
   ...:

In [4]: a = A()

In [5]: b = B()

In [6]: id(a.test)
Out[6]: 140631076810816

In [7]: id(b.test)
Out[7]: 140631076791872

Environment:

System:
Ubuntu 24.04 in wsl2 under Windows 11
Also reproduces in original Ubuntu 24.04

(venv) [21:30:49] check_pytest_issue $ uname -a
Linux dllwn 5.15.167.4-microsoft-standard-WSL2 #1 SMP Tue Nov 5 00:21:55 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Python & packages:

(venv) [21:30:52] check_pytest_issue $ python --version
Python 3.12.3
(venv) [21:31:24] check_pytest_issue $ pip list
Package               Version
--------------------- -------
asttokens             3.0.0
decorator             5.1.1
docstring-to-markdown 0.15
executing             2.1.0
iniconfig             2.0.0
ipdb                  0.13.13
ipython               8.30.0
jedi                  0.19.2
matplotlib-inline     0.1.7
packaging             24.2
parso                 0.8.4
pexpect               4.9.0
pip                   24.0
pluggy                1.5.0
prompt_toolkit        3.0.48
ptyprocess            0.7.0
pure_eval             0.2.3
Pygments              2.18.0
pytest                8.3.4
python-lsp-jsonrpc    1.1.2
python-lsp-server     1.12.0
ruff                  0.8.4
stack-data            0.6.3
traitlets             5.14.3
ujson                 5.10.0
wcwidth               0.2.13

[RonnyPfannschmidt]

this is expected - and a user error

direct modification of a function object will assure the net time its collected from a different class

[201dreamers]

okay, thanks for such a quick response!
Could you please tell me how can I properly add marks in such cases, so the mark isn't applied to all cases? (Of course if you have time. If not, I'll try to look myself)

[RonnyPfannschmidt]

more context is needed

the minimal example you posted is best served by using the mark as decorator in the target class

[201dreamers]

What about the case when I need to add marks from the pytest_generate_tests hook, and the cases are different, some of them are class-based, some function-based?

[RonnyPfannschmidt]

show the code - generae tests is supposed to use pytest.param(..., marks=...) for adding per case markers

[201dreamers]

It's hard to do, because the code from the project i'm working with is pretty complicated and I don't know how to simplify it for an example here without violating an nda. Anyway you showed me that this is the misuse of metafunc.function object, so i'll try to figure out how to fix it in our framework with the team. Since it's not a bug, from my side I think this issue can be closed. I'll create a question, when I have any. Thank you for your time and responses!

[github-actions]

This issue is stale because it has the status: needs information label and requested follow-up information was not provided for 14 days.