Slicer Python Test Runner

Table of contents

• Introduction
• Why this extension
• Using the extension
• Test decorators
• Changelog
• Contributing

Introduction

This module allows running 3D Slicer module's unit tests directly from 3D Slicer's UI.

It uses PyTest and PyTest JSON Report to discover the unit tests in a given directory or file and runs the tests in a separate 3D Slicer process.

After the tests have been run, the results are displayed in the UI. The plugin also provides decorators to help running unit tests directly in your favorite IDEs.

Why this extension

3D Slicer is a C++ based application and provides numerous libraries which are conveniently wrapped and available in its Python environment.

The Python interpreter is named python-real in Slicer's binary folder and can be used directly to run Python libraries installed in the Python environment such as NumPy and ITK.

Some features, however, are only available when running inside the full 3D Slicer environment, for instance access to the application main window or accessing loaded modules.

When testing a 3D Slicer modules, it is hence necessary to run in the full 3D Slicer environment.

3D Slicer provides a Reload and Test developer tool which reloads the current module python file and runs its unit tests. However, this mechanism has the following limitations :

• Due to Python's lack of reloading mechanism for packages, it only works well for single file modules
• The tests run only provides print in Slicer's console
• The test run doesn't allow for filtering tests to run

3D Slicer also has CTest testing functionalities. However, these functionalities are not readily available for a developer wishing to develop a Slicer extension using a pre-built application.

By launching a new Slicer process for its tests and using PyTest's filtering mechanism, this extension provides an efficient way of running UnitTests inside 3D Slicer without the previous caveats.

Using the extension

This extension can be installed directly using Slicer's extension manager.

Once installed, navigate to Developer Tools>Slicer Python Test Runner in the modules drop down menu or search directly for Slicer Python Test Runner.

You will see the following, empty, test runner module:

Start by selecting the directory containing the tests using the top path selector. Then enter the file filter pattern, and name filter pattern. Note: The function filter will be applied when running the tests but not during test collection.

Warning : Note that by default, the module uses PyTest's conventions which expects test files, classes and functions to be prefixed by test.

If you are testing using 3D Slicer convention, please use the following pattern:

• file : *.py

Click on the collect button to check that the tests to run are correctly found by the extension.

Afterwards, click on the run button to start testing.

You can cancel test running using the stop button. Note however that as the test runner starts new Slicer processes the cancel may not be graceful and 3D Slicer windows may have to be killed manually.

Once the tests have run, you will see the tests results in the UI.

Clicking on one test in the tree view will display its result in the text editor below, along the error if any.

Additional buttons allow to filter the tests displayed in the UI (passed, ignored, collected).

To modify the execution settings, click on the settings button.

The following settings are accessible:

• Close Slicer after run: Closes the spawned Slicer process when the tests have finished running.
• Use main window: If unchecked, opens the Slicer process without any main window GUI
• Minimize main window: If checked and a main window is used, minimizes the launched window at startup
• Extra slicer args: Comma separated list of args to use when starting Slicer instance (refer to Slicer launcher CLI args for more info)
• Extra pytest args: Comma separated list of args to pass to PyTest (please refer to PyTest args for more info)
• Run test coverage: If checked, test coverage will be executed on the tests which are ran
• Coverage report format: Comma separated list of coverage formats. For now only one format should be used from the UI. If you want to output more coverage files, it is recommended to use coverage settings file in your project.
• Coverage source: Comma separated list of sources folders to include in the coverage report.
• Coverage path: Output file or directory path where the coverage report will be put.

The test module is also compatible with pytest.ini and .coveragerc settings files. It should also be compatible with other settings files in your project to ease the configuration process.

Test decorators

This module also provides the following decorators :

• runTestInSlicerContext: This decorator will run the current test by using the Module's test runner if executed outside a 3D Slicer application. This allows to run the tests directly in your favorite IDE's test runner.
• skipTestOutsideSlicer: This decorator will skip the current test if executed outside a 3D Slicer application.

Changelog

Contributing

This project welcomes contributions. If you want more information about how you can contribute, please refer to the CONTRIBUTING.md file.