mp plugin has issues with unittest.skip

[twizmwazin]

I seem to have come across a weird issue when using unittest.skip on individual methods in a unittest.TestCase in combination with the nose2.plugins.mp plugin. Here is an example to show my issue:

Code (test_ex.py):

import unittest

class MyTest(unittest.TestCase):
    @unittest.skip("aaa")
    def test_something(self):
        pass

Running without mp (correct behaviour):

$ nose2 -s . -v
test_something (test_ex.MyTest) ... skipped aaa

----------------------------------------------------------------------
Ran 1 test in 0.000s

OK (skipped=1)

Running with mp:

$ nose2 -s . -v --plugin nose2.plugins.mp -N1
test_not_found (nose2.loader.LoadTestsFailure) ... skipped aaa

----------------------------------------------------------------------
Ran 1 test in 0.005s

OK (skipped=1)

And one more example, if I comment out the unittest.skip decorator:

$ nose2 -s . -v --plugin nose2.plugins.mp -N1
test_something (test_ex.MyTest) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.005s

OK

When the mp plugin is enabled and the skip decorator is used, the test is incorrectly labelled. In addition to the misreporting, this also is an issue when naming specific tests to run:

$ nose2 -s . -v --plugin nose2.plugins.mp -N1 test_ex.MyTest.test_something
test_not_found (nose2.loader.LoadTestsFailure) ... ERROR

======================================================================
ERROR: test_not_found (nose2.loader.LoadTestsFailure)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kevin/.virtualenvs/angr/lib/python3.7/site-packages/nose2/util.py", line 165, in try_import_module_from_name
    module = __import__('.'.join(splitted_name))
ModuleNotFoundError: No module named 'test_ex.test_something'; 'test_ex' is not a package

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/kevin/.virtualenvs/angr/lib/python3.7/site-packages/six.py", line 693, in reraise
    raise value
  File "/home/kevin/.virtualenvs/angr/lib/python3.7/site-packages/nose2/plugins/loader/functions.py", line 81, in loadTestsFromName
    result = util.test_from_name(name, module)
  File "/home/kevin/.virtualenvs/angr/lib/python3.7/site-packages/nose2/util.py", line 93, in test_from_name
    parent, obj = object_from_name(name, module)
  File "/home/kevin/.virtualenvs/angr/lib/python3.7/site-packages/nose2/util.py", line 124, in object_from_name
    _raise_custom_attribute_error(obj, part, e, import_error)
  File "/home/kevin/.virtualenvs/angr/lib/python3.7/site-packages/nose2/util.py", line 134, in _raise_custom_attribute_error
    six.raise_from(attr_error_exc, prev_exc[1])
  File "<string>", line 3, in raise_from
  File "/home/kevin/.virtualenvs/angr/lib/python3.7/site-packages/nose2/util.py", line 119, in object_from_name
    parent, obj = obj, getattr(obj, part)
AttributeError: module 'test_ex' has no attribute 'test_something'

-------------------- >> begin captured stdout << ---------------------

--------------------- >> end captured stdout << ----------------------
----------------------------------------------------------------------
Ran 1 test in 0.006s

FAILED (errors=1)

It seems to have cut out the TestCase class name and is failing as a result, unsure if this is related to #42. Any help or suggestions would be greatly appreciated!

[sirosen]

Could you let me know which python version you're using, and which version of nose2?

I wasn't able to reproduce using the latest release on my installed python versions.

[twizmwazin]

Hey! Sorry for taking a while to reply here, It turns out that simply loading the plugin isn't enough to cause the bug, you also need to actually activate the mp plugin. Setting always on in the config, or -N1 is sufficient to reproduce the issue in a python:3 docker container (currently python 3.8.2) for the version 0.9.2 from pypi.

[stefanholek]

Hi,

While working on other issues (#468, #444) I came up with PR #472 and it appears to fix the decoration problem with unittest.skip just as it does with mock.patch.

The proposed PR #466 somewhat misdiagnoses the bug: It is not about the function loader getting the name wrong, it is about the function loader acting on class methods in the first place! Class methods are the testcase loader's responsibility!

[ltfish]

Class methods are the testcase loader's responsibility!

Good to know! Thanks for working on addressing this issue.

[sirosen]

With #472 merged, I'm going to close this. In the upcoming release, this should be fixed.
Kudos to @stefanholek for figuring this out!