pytest.mark.parametrize ids with empty parameter

[martinsvat]

a detailed description of the bug or problem you are having, MWE

I have parametrized tests that take input data from a config, e.g., a list of matrices. It may happen that there is no input parameter at all, i.e., empty list of parameters, see below

import pytest

parameters = []


@pytest.mark.parametrize(
    "matrix",
    parameters,
    ids=lambda matrix: matrix.id,
)
def test_foo(matrix):
    return True

Now, I'd like to just collect matrices there are, i.e.
pytest dev.py --collect-only
but it fails as it tries to create at least a single instance for test_foo. The reason is that ids is a callable and it's called with NotSetType.

===================================================================================================================================================== test session starts =====================================================================================================================================================
platform linux -- Python 3.11.3, pytest-8.3.3, pluggy-1.5.0
benchmark: 5.1.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: ...
configfile: ...
plugins: monitor-1.6.1, benchmark-5.1.0
collected 0 items / 1 error                                                                                                                                                                                                                                                                                                   

=========================================================================================================================================================== ERRORS ============================================================================================================================================================
___________________________________________________________________________________________________________________________________________________ ERROR collecting dev.py ___________________________________________________________________________________________________________________________________________________
dev.py:9: in <lambda>
    ids=lambda matrix: matrix.id,
E   AttributeError: 'NotSetType' object has no attribute 'id'

The above exception was the direct cause of the following exception:
.venv/lib/python3.11/site-packages/pluggy/_hooks.py:513: in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
.venv/lib/python3.11/site-packages/pluggy/_manager.py:120: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
.venv/lib/python3.11/site-packages/_pytest/python.py:245: in pytest_pycollect_makeitem
    return list(collector._genfunctions(name, obj))
.venv/lib/python3.11/site-packages/_pytest/python.py:462: in _genfunctions
    self.ihook.pytest_generate_tests.call_extra(methods, dict(metafunc=metafunc))
.venv/lib/python3.11/site-packages/pluggy/_hooks.py:574: in call_extra
    return self._hookexec(self.name, hookimpls, kwargs, firstresult)
.venv/lib/python3.11/site-packages/pluggy/_manager.py:120: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
.venv/lib/python3.11/site-packages/_pytest/python.py:115: in pytest_generate_tests
    metafunc.parametrize(*marker.args, **marker.kwargs, _param_mark=marker)
.venv/lib/python3.11/site-packages/_pytest/python.py:1236: in parametrize
    ids = self._resolve_parameter_set_ids(
.venv/lib/python3.11/site-packages/_pytest/python.py:1355: in _resolve_parameter_set_ids
    return id_maker.make_unique_parameterset_ids()
.venv/lib/python3.11/site-packages/_pytest/python.py:897: in make_unique_parameterset_ids
    resolved_ids = list(self._resolve_ids())
.venv/lib/python3.11/site-packages/_pytest/python.py:932: in _resolve_ids
    yield "-".join(
.venv/lib/python3.11/site-packages/_pytest/python.py:933: in <genexpr>
    self._idval(val, argname, idx)
.venv/lib/python3.11/site-packages/_pytest/python.py:939: in _idval
    idval = self._idval_from_function(val, argname, idx)
.venv/lib/python3.11/site-packages/_pytest/python.py:961: in _idval_from_function
    raise ValueError(msg) from e
E   ValueError: dev.py::test_foo: error raised while trying to determine id of parameter 'matrix' at position 0
=================================================================================================================================================== short test summary info ===================================================================================================================================================
ERROR dev.py - ValueError: dev.py::test_foo: error raised while trying to determine id of parameter 'matrix' at position 0
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

I found a nasty workaround

import pytest

parameters = []


@pytest.mark.parametrize(
    "matrix",
    parameters,
    ids=(lambda matrix: matrix.id) if parameters else None,
)
def test_foo(matrix):
    return True

which returns

...
plugins: monitor-1.6.1, benchmark-5.1.0
collected 1 item                                                                                                                                                                                                                                                                                                              

<Dir bigplan-algorithm>
  <Module dev.py>
    <Function test_foo[matrix0]>
...

While the problem might seem a bit artificial, it's a scenario in my case. I have no control on what data are in the config, I just need to test the thing without failures. The workaround is doable but I would don't like writing these for each single case, and frankly, don't understand why ids are invoked at all.

output of pip list from the virtual environment you are using

igplan-algorithm>
  <Module dev.py>
    <Function test_foo[matrix0]>

================================================================================================================================================== 1 test collected in 0.01s ==================================================================================================================================================
> pip list
Package                  Version
------------------------ --------------
pytest                   8.3.3
pytest-benchmark         5.1.0
pytest-monitor           1.6.1
pytest-xdist             3.6.1

20.04.1-Ubuntu

[The-Compiler]

Even if maybe somewhat artificial, given that pytest._compat.NOTSET is a private object and undocumented, it seems like a bug for pytest to expose it like that. From an user's perspective, I'd expect the id-function to not be called at all in this case.

From a cursory look at the code, the plugin hook seems to be affected in a similar way:

import pytest

@pytest.mark.parametrize("a", [])
def test_nothing(a):
    pass

conftest.py:

def pytest_make_parametrize_id(val):
    assert False, val

results in:

E   AssertionError: <NotSetType.token: 0>

[RonnyPfannschmidt]

Fixed in #13073

[RonnyPfannschmidt]

I just took note that the bsckport failed

Let's close this one via the manuak backport

https://github.com/pytest-dev/pytest/issues/13031is the original issue