TESTING.rst

Testing with Octavia

Unit Testing

Octavia uses tox to manage the virtual environments for running test cases.

Install python-tox:

$ pip install tox

To run the full suite of tests maintained within Octavia.

$ tox

Note

The first time you run tox, it will take additional time to build virtualenvs. You can later use the -r option with tox to rebuild your virtualenv in a similar manner.

To run tests for one or more specific test environments(for example, the most common configuration of Python 3.8 and PEP-8), list the environments with the -e option, separated by spaces:

$ tox -e py38,pep8

See tox -l for the full list of available test environments.

Structure of the Unit Test Tree

The structure of the unit test tree should match the structure of the code tree, e.g.

- target module: octavia.common.utils

- test module: octavia.tests.unit.common.test_utils

Unit test modules should have the same path under octavia/tests/unit/ as the module they target has under octavia/, and their name should be the name of the target module prefixed by test_. This requirement is intended to make it easier for developers to find the unit tests for a given module.

Similarly, when a test module targets a package, that module's name should be the name of the package prefixed by test_ with the same path as when a test targets a module, e.g.

- target package: octavia.hacking

- test module: octavia.tests.unit.test_hacking

The following command can be used to validate whether the unit test tree is structured according to the above requirements:

./tools/check_unit_test_structure.sh

Where appropriate, exceptions can be added to the above script. If code is not part of the Octavia namespace, for example, it's probably reasonable to exclude their unit tests from the check.

Functional Testing

Octavia creates a simulated API and handler for its functional tests. The tests then run requests against the mocked up API.

To run the entire suite of functional tests:

$ tox -e functional

To run a specific functional test:

$ tox -e functional octavia.tests.functional.api.v2.test_load_balancer

Tests can also be run using partial matching, to run all API tests for v2:

$ tox -e functional api.v2

Additional options can be used while running tests. Two useful options that can be used when running tests are -- --until-failure which will run the tests in a loop until the first failure is hit, and -- --failing which if used after an initial run will only run the tests that failed in the previous run.

Scenario Testing

Octavia uses Tempest to cover the scenario tests for the project. These tests are run against actual cloud deployments.

To run the entire suite of scenario tests:

$ tox -e scenario

Note

The first time running the Tempest scenario tests export the Tempest configuration directory (i.e. TEMPEST_CONFIG_DIR=/opt/stack/tempest/etc)