Merge pull request #6716 from bluetech/features-to-master-for-real

Merge the features branch into master, before stopping to use it

.pre-commit-config.yaml

@@ -53,8 +53,8 @@ repos:
     -   id: changelogs-rst
         name: changelog filenames
         language: fail
-        entry: 'changelog files must be named ####.(feature|bugfix|doc|deprecation|removal|vendor|trivial).rst'
-        exclude: changelog/(\d+\.(feature|improvement|bugfix|doc|deprecation|removal|vendor|trivial).rst|README.rst|_template.rst)
+        entry: 'changelog files must be named ####.(breaking|bugfix|deprecation|doc|feature|improvement|trivial|vendor).rst'
+        exclude: changelog/(\d+\.(breaking|bugfix|deprecation|doc|feature|improvement|trivial|vendor).rst|README.rst|_template.rst)
         files: ^changelog/
     -   id: py-deprecated
         name: py library is deprecated

AUTHORS

@@ -52,20 +52,21 @@ Carl Friedrich Bolz
 Carlos Jenkins
 Ceridwen
 Charles Cloud
+Charles Machalow
 Charnjit SiNGH (CCSJ)
 Chris Lamb
 Christian Boelsen
 Christian Fetzer
 Christian Neumüller
 Christian Theunert
 Christian Tismer
-Christopher Gilling
+Christoph Buelter
 Christopher Dignam
+Christopher Gilling
 Claudio Madotto
 CrazyMerlyn
 Cyrus Maden
 Damian Skrzypczak
-Dhiren Serai
 Daniel Grana
 Daniel Hahler
 Daniel Nuri
@@ -80,6 +81,7 @@ David Szotten
 David Vierra
 Daw-Ran Liou
 Denis Kirisov
+Dhiren Serai
 Diego Russo
 Dmitry Dygalo
 Dmitry Pribysh
@@ -121,6 +123,7 @@ Ilya Konstantinov
 Ionuț Turturică
 Iwan Briquemont
 Jaap Broekhuizen
+Jakub Mitoraj
 Jan Balster
 Janne Vanhala
 Jason R. Coombs
@@ -207,8 +210,10 @@ Omer Hadari
 Ondřej Súkup
 Oscar Benjamin
 Patrick Hayes
+Pauli Virtanen
 Paweł Adamczak
 Pedro Algarvio
+Philipp Loose
 Pieter Mulder
 Piotr Banaszkiewicz
 Pulkit Goyal
@@ -269,6 +274,7 @@ Vidar T. Fauske
 Virgil Dupras
 Vitaly Lashmanov
 Vlad Dragos
+Vladyslav Rachek
 Volodymyr Piskun
 Wei Lin
 Wil Cooley

changelog/1857.improvement.rst

@@ -0,0 +1 @@
+``pytest.mark.parametrize`` accepts integers for ``ids`` again, converting it to strings.

changelog/3238.deprecation.rst

@@ -0,0 +1,5 @@
+Option ``--no-print-logs`` is deprecated and meant to be removed in a future release. If you use ``--no-print-logs``, please try out ``--show-capture`` and
+provide feedback.
+
+``--show-capture`` command-line option was added in ``pytest 3.5.0`` and allows to specify how to
+display captured output when tests fail: ``no``, ``stdout``, ``stderr``, ``log`` or ``all`` (the default).

changelog/3823.bugfix.rst

@@ -0,0 +1 @@
+``--trace`` now works with unittests.

changelog/4445.bugfix.rst

@@ -0,0 +1 @@
+Fixed some warning reports produced by pytest to point to the correct location of the warning in the user's code.

changelog/449.improvement.rst

@@ -0,0 +1 @@
+Use "yellow" main color with any XPASSED tests.

changelog/4597.feature.rst

@@ -0,0 +1 @@
+New :ref:`--capture=tee-sys <capture-method>` option to allow both live printing and capturing of test output.

changelog/4639.improvement.rst

@@ -0,0 +1,4 @@
+Revert "A warning is now issued when assertions are made for ``None``".
+
+The warning proved to be less useful than initially expected and had quite a
+few false positive cases.

changelog/5686.improvement.rst

@@ -0,0 +1 @@
+``tmpdir_factory.mktemp`` now fails when given absolute and non-normalized paths.

changelog/5712.feature.rst

@@ -0,0 +1,2 @@
+Now all arguments to ``@pytest.mark.parametrize`` need to be explicitly declared in the function signature or via ``indirect``.
+Previously it was possible to omit an argument if a fixture with the same name existed, which was just an accident of implementation and was not meant to be a part of the API.

changelog/5928.bugfix.rst

@@ -0,0 +1 @@
+Report ``PytestUnknownMarkWarning`` at the level of the user's code, not ``pytest``'s.

changelog/5975.deprecation.rst

@@ -0,0 +1,10 @@
+Deprecate using direct constructors for ``Nodes``.
+
+Instead they are new constructed via ``Node.from_parent``.
+
+This transitional mechanism enables us to detangle the very intensely
+entangled ``Node`` relationships by enforcing more controlled creation/configruation patterns.
+
+As part of that session/config are already disallowed parameters and as we work on the details we might need disallow a few more as well.
+
+Subclasses are expected to use `super().from_parent` if they intend to expand the creation of `Nodes`.

changelog/5984.improvement.rst

@@ -0,0 +1 @@
+The ``pytest_warning_captured`` hook now receives a ``location`` parameter with the code location that generated the warning.

changelog/5991.bugfix.rst

@@ -0,0 +1 @@
+Fix interaction with ``--pdb`` and unittests: do not use unittest's ``TestCase.debug()``.

changelog/6213.improvement.rst

@@ -0,0 +1 @@
+pytester: the ``testdir`` fixture respects environment settings from the ``monkeypatch`` fixture for inner runs.

changelog/6247.improvement.rst

@@ -0,0 +1 @@
+``--fulltrace`` is honored with collection errors.

changelog/6316.breaking.rst

@@ -0,0 +1 @@
+Matching of ``-k EXPRESSION`` to test names is now case-insensitive.

changelog/6334.bugfix.rst

@@ -0,0 +1,3 @@
+Fix summary entries appearing twice when ``f/F`` and ``s/S`` report chars were used at the same time in the ``-r`` command-line option (for example ``-rFf``).
+
+The upper case variants were never documented and the preferred form should be the lower case.

changelog/6384.improvement.rst

@@ -0,0 +1 @@
+Make `--showlocals` work also with `--tb=short`.

changelog/6404.trivial.rst

@@ -0,0 +1 @@
+Remove usage of ``parser`` module, deprecated in Python 3.9.

changelog/6443.breaking.rst

@@ -0,0 +1,3 @@
+Plugins specified with ``-p`` are now loaded after internal plugins, which results in their hooks being called *before* the internal ones.
+
+This makes the ``-p`` behavior consistent with ``PYTEST_PLUGINS``.

changelog/6454.bugfix.rst

@@ -0,0 +1 @@
+`--disable-warnings` is honored with `-ra` and `-rA`.

changelog/6454.feature.rst

@@ -0,0 +1 @@
+Changed default for `-r` to `fE`, which displays failures and errors in the :ref:`short test summary <pytest.detailed_failed_tests_usage>`.  `-rN` can be used to disable it (the old behavior).

changelog/6469.feature.rst

@@ -0,0 +1 @@
+New options have been added to the :confval:`junit_logging` option: ``log``, ``out-err``, and ``all``.

changelog/6597.bugfix.rst

@@ -0,0 +1 @@
+Fix node ids which contain a parametrized empty-string variable.

changelog/6637.breaking.rst

@@ -0,0 +1,3 @@
+Removed the long-deprecated ``pytest_itemstart`` hook.
+
+This hook has been marked as deprecated and not been even called by pytest for over 10 years now.

changelog/6653.improvement.rst

@@ -0,0 +1 @@
+Add support for matching lines consecutively with :attr:`LineMatcher <_pytest.pytester.LineMatcher>`'s :func:`~_pytest.pytester.LineMatcher.fnmatch_lines` and :func:`~_pytest.pytester.LineMatcher.re_match_lines`.

changelog/6658.improvement.rst

@@ -0,0 +1,4 @@
+Code is now highlighted in tracebacks when ``pygments`` is installed.
+
+Users are encouraged to install ``pygments`` into their environment and provide feedback, because
+the plan is to make ``pygments`` a regular dependency in the future.

changelog/759.improvement.rst

@@ -0,0 +1 @@
+``pytest.mark.parametrize`` supports iterators and generators for ``ids``.

changelog/README.rst

@@ -18,7 +18,7 @@ Each file should be named like ``<ISSUE>.<TYPE>.rst``, where
 * ``bugfix``: fixes a bug.
 * ``doc``: documentation improvement, like rewording an entire session or adding missing docs.
 * ``deprecation``: feature deprecation.
-* ``removal``: feature removal.
+* ``breaking``: a change which may break existing suites, such as feature removal or behavior change.
 * ``vendor``: changes in packages vendored in pytest.
 * ``trivial``: fixing a small typo or internal change that might be noteworthy.
 

doc/en/backwards-compatibility.rst

@@ -3,6 +3,61 @@
 Backwards Compatibility Policy
 ==============================
 
+.. versionadded: 6.0
+
+pytest is actively evolving and is a project that has been decades in the making,
+we keep learning about new and better structures to express different details about testing.
+
+While we implement those modifications we try to ensure an easy transition and don't want to impose unnecessary churn on our users and community/plugin authors.
+
+As of now, pytest considers multipe types of backward compatibility transitions:
+
+a) trivial: APIs which trivially translate to the new mechanism,
+   and do not cause problematic changes.
+
+   We try to support those indefinitely while encouraging users to switch to newer/better mechanisms through documentation.
+
+b) transitional: the old and new API don't conflict
+   and we can help users transition by using warnings, while supporting both for a prolonged time.
+
+   We will only start the removal of deprecated functionality in major releases (e.g. if we deprecate something in 3.0 we will start to remove it in 4.0), and keep it around for at least two minor releases (e.g. if we deprecate something in 3.9 and 4.0 is the next release, we start to remove it in 5.0, not in 4.0).
+
+   When the deprecation expires (e.g. 4.0 is released), we won't remove the deprecated functionality immediately, but will use the standard warning filters to turn them into **errors** by default. This approach makes it explicit that removal is imminent, and still gives you time to turn the deprecated feature into a warning instead of an error so it can be dealt with in your own time. In the next minor release (e.g. 4.1), the feature will be effectively removed.
+
+
+c) true breakage: should only to be considered when normal transition is unreasonably unsustainable and would offset important development/features by years.
+   In addition, they should be limited to APIs where the number of actual users is very small (for example only impacting some plugins), and can be coordinated with the community in advance.
+
+   Examples for such upcoming changes:
+
+   * removal of ``pytest_runtest_protocol/nextitem`` - `#895`_
+   * rearranging of the node tree to include ``FunctionDefinition``
+   * rearranging of ``SetupState`` `#895`_
+
+   True breakages must be announced first in an issue containing:
+
+   * Detailed description of the change
+   * Rationale
+   * Expected impact on users and plugin authors (example in `#895`_)
+
+   After there's no hard *-1* on the issue it should be followed up by an initial proof-of-concept Pull Request.
+
+   This POC serves as both a coordination point to assess impact and potential inspriation to come up with a transitional solution after all.
+
+   After a reasonable amount of time the PR can be merged to base a new major release.
+
+   For the PR to mature from POC to acceptance, it must contain:
+   * Setup of deprecation errors/warnings that help users fix and port their code. If it is possible to introduce a deprecation period under the current series, before the true breakage, it should be introduced in a separate PR and be part of the current release stream.
+   * Detailed description of the rationale and examples on how to port code in ``doc/en/deprecations.rst``.
+
+
+History
+=========
+
+
+Focus primary on smooth transition - stance (pre 6.0)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
 Keeping backwards compatibility has a very high priority in the pytest project. Although we have deprecated functionality over the years, most of it is still supported. All deprecations in pytest were done because simpler or more efficient ways of accomplishing the same tasks have emerged, making the old way of doing things unnecessary.
 
 With the pytest 3.0 release we introduced a clear communication scheme for when we will actually remove the old busted joint and politely ask you to use the new hotness instead, while giving you enough time to adjust your tests or raise concerns if there are valid reasons to keep deprecated functionality around.
@@ -20,3 +75,6 @@ Deprecation Roadmap
 Features currently deprecated and removed in previous releases can be found in :ref:`deprecations`.
 
 We track future deprecation and removal of features using milestones and the `deprecation <https://github.com/pytest-dev/pytest/issues?q=label%3A%22type%3A+deprecation%22>`_ and `removal <https://github.com/pytest-dev/pytest/labels/type%3A%20removal>`_ labels on GitHub.
+
+
+.. _`#895`: https://github.com/pytest-dev/pytest/issues/895

doc/en/capture.rst

@@ -21,27 +21,36 @@ file descriptors.  This allows to capture output from simple
 print statements as well as output from a subprocess started by
 a test.
 
+.. _capture-method:
+
 Setting capturing methods or disabling capturing
 -------------------------------------------------
 
-There are two ways in which ``pytest`` can perform capturing:
+There are three ways in which ``pytest`` can perform capturing:
 
-* file descriptor (FD) level capturing (default): All writes going to the
+* ``fd`` (file descriptor) level capturing (default): All writes going to the
   operating system file descriptors 1 and 2 will be captured.
 
 * ``sys`` level capturing: Only writes to Python files ``sys.stdout``
   and ``sys.stderr`` will be captured.  No capturing of writes to
   filedescriptors is performed.
 
+* ``tee-sys`` capturing: Python writes to ``sys.stdout`` and ``sys.stderr``
+  will be captured, however the writes will also be passed-through to
+  the actual ``sys.stdout`` and ``sys.stderr``. This allows output to be
+  'live printed' and captured for plugin use, such as junitxml (new in pytest 5.4).
+
 .. _`disable capturing`:
 
 You can influence output capturing mechanisms from the command line:
 
 .. code-block:: bash
 
-    pytest -s            # disable all capturing
-    pytest --capture=sys # replace sys.stdout/stderr with in-mem files
-    pytest --capture=fd  # also point filedescriptors 1 and 2 to temp file
+    pytest -s                  # disable all capturing
+    pytest --capture=sys       # replace sys.stdout/stderr with in-mem files
+    pytest --capture=fd        # also point filedescriptors 1 and 2 to temp file
+    pytest --capture=tee-sys   # combines 'sys' and '-s', capturing sys.stdout/stderr
+                               # and passing it along to the actual sys.stdout/stderr
 
 .. _printdebugging:
 

doc/en/deprecations.rst

@@ -19,6 +19,30 @@ Below is a complete list of all pytest features which are considered deprecated.
 :class:`_pytest.warning_types.PytestWarning` or subclasses, which can be filtered using
 :ref:`standard warning filters <warnings>`.
 
+
+``--no-print-logs`` command-line option
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. deprecated:: 5.4
+
+
+Option ``--no-print-logs`` is deprecated and meant to be removed in a future release. If you use ``--no-print-logs``, please try out ``--show-capture`` and
+provide feedback.
+
+``--show-capture`` command-line option was added in ``pytest 3.5.0` and allows to specify how to
+display captured output when tests fail: ``no``, ``stdout``, ``stderr``, ``log`` or ``all`` (the default).
+
+
+
+Node Construction changed to ``Node.from_parent``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. deprecated:: 5.3
+
+The construction of nodes new should use the named constructor ``from_parent``.
+This limitation in api surface intends to enable better/simpler refactoring of the collection tree.
+
+
 ``junit_family`` default value change to "xunit2"
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

doc/en/example/markers.rst

@@ -148,6 +148,10 @@ which implements a substring match on the test names instead of the
 exact match on markers that ``-m`` provides.  This makes it easy to
 select tests based on their names:
 
+.. versionadded: 5.4
+
+The expression matching is now case-insensitive.
+
 .. code-block:: pytest
 
     $ pytest -v -k http  # running with the above defined example module

doc/en/example/nonpython/conftest.py

@@ -4,7 +4,7 @@
 
 def pytest_collect_file(parent, path):
     if path.ext == ".yaml" and path.basename.startswith("test"):
-        return YamlFile(path, parent)
+        return YamlFile.from_parent(parent, fspath=path)
 
 
 class YamlFile(pytest.File):
@@ -13,7 +13,7 @@ def collect(self):
 
         raw = yaml.safe_load(self.fspath.open())
         for name, spec in sorted(raw.items()):
-            yield YamlItem(name, self, spec)
+            yield YamlItem.from_parent(self, name=name, spec=spec)
 
 
 class YamlItem(pytest.Item):

doc/en/example/parametrize.rst

@@ -398,6 +398,9 @@ The result of this test will be successful:
 
 .. regendoc:wipe
 
+Note, that each argument in `parametrize` list should be explicitly declared in corresponding
+python test function or via `indirect`.
+
 Parametrizing test methods through per-class configuration
 --------------------------------------------------------------
 

doc/en/pythonpath.rst

@@ -77,6 +77,8 @@ This is also discussed in details in :ref:`test discovery`.
 Invoking ``pytest`` versus ``python -m pytest``
 -----------------------------------------------
 
-Running pytest with ``python -m pytest [...]`` instead of ``pytest [...]`` yields nearly
-equivalent behaviour, except that the former call will add the current directory to ``sys.path``.
+Running pytest with ``pytest [...]`` instead of ``python -m pytest [...]`` yields nearly
+equivalent behaviour, except that the latter will add the current directory to ``sys.path``, which
+is standard ``python`` behavior.
+
 See also :ref:`cmdline`.

doc/en/reference.rst

@@ -738,7 +738,7 @@ ExceptionInfo
 pytest.ExitCode
 ~~~~~~~~~~~~~~~
 
-.. autoclass:: _pytest.main.ExitCode
+.. autoclass:: _pytest.config.ExitCode
     :members:
 
 
@@ -1164,9 +1164,17 @@ passed multiple times. The expected format is ``name=value``. For example::
 .. confval:: junit_logging
 
     .. versionadded:: 3.5
+    .. versionchanged:: 5.4
+        ``log``, ``all``, ``out-err`` options added.
 
-    Configures if stdout/stderr should be written to the JUnit XML file. Valid values are
-    ``system-out``, ``system-err``, and ``no`` (the default).
+    Configures if captured output should be written to the JUnit XML file. Valid values are:
+
+    * ``log``: write only ``logging`` captured output.
+    * ``system-out``: write captured ``stdout`` contents.
+    * ``system-err``: write captured ``stderr`` contents.
+    * ``out-err``: write both captured ``stdout`` and ``stderr`` contents.
+    * ``all``: write captured ``logging``, ``stdout`` and ``stderr`` contents.
+    * ``no`` (the default): no captured output is written.
 
     .. code-block:: ini
 

doc/en/unittest.rst

@@ -238,17 +238,6 @@ was executed ahead of the ``test_method``.
 
 .. _pdb-unittest-note:
 
-.. note::
-
-    Running tests from ``unittest.TestCase`` subclasses with ``--pdb`` will
-    disable tearDown and cleanup methods for the case that an Exception
-    occurs. This allows proper post mortem debugging for all applications
-    which have significant logic in their tearDown machinery. However,
-    supporting this feature has the following side effect: If people
-    overwrite ``unittest.TestCase`` ``__call__`` or ``run``, they need to
-    to overwrite ``debug`` in the same way  (this is also true for standard
-    unittest).
-
 .. note::
 
     Due to architectural differences between the two frameworks, setup and