Test Suite

Automated tests (continuous integration) are run on Travis and Microsoft Azure. These tests are run on pull requests and branches, and are required to pass before branches will be merged!

To run the tests locally, you can run:

# Optionally set an env variable run test jobs in parallel (much faster)
export MAKEFLAGS=-j6

# Run all tests
make test

# Run one specific test
make test TESTSRCS=tests/absmin.xml

# Run a couple select tests
make test TESTSRCS="tests/absmin.xml tests/eject.sil"

The test suite is somewhat clever in how caches its results and dependencies, meaning that it only re-runs tests which have been affected by source changes. The test suite also downloads any fonts that it needs, storing them in the .fonts directory.

To force the suite to ignore its (sometimes too smart) dependency checking and rerun a test anyway, set an environment variable CLEAN to true or append it to the make invocation:

make test CLEAN=true

It can also produce preview PDFs of the test files for visual inspection; to rebuild these, use

make regression_previews
make regression_previews TESTSRCS=tests/absmin.xml

To manually fix up a test expectation (declaring, after you've inspected the PDF) that the output is acceptable, regenerate it and commit the result in a dedicated commit:

make tests/test_name.expected
git add tests/test_name.expected
git commit -m "test(<component>): Update expected output of test_name"