Insist on luatest

[Totktonada]

Tarantool test suite contains tests written using different approaches.

Four test types in test-run (source)

In test-run (the test framework that we use in tarantool/tarantool) we have four types of tests (named below according to the core field in the suite.ini file):

• core = tarantool: read a test file and feed it line-by-line¹ to tarantool console. A text of requests and responses is squashed and compared against a reference file. Often called diff-based based testing.
• core = app: just Lua script. Usually produces TAP13 output using built-in tap module (and so often called tap-based testing), which is validated by test-run without a reference file. If a test produces non-TAP13 output, stdout is compared against a reference file.
• core = unittest: just executable files. May be TAP13 compliant (after some work) or diff based.
• core = luatest: calls luatest <...> /path/to/foo_test.lua, where luatest is configured to produce TAP13 output. (Reasoning.)

Some time ago we agreed to write new tests using the luatest framework, because it allows to make the testing code more structed using explicit asserts and before/after test hooks.

This agreement is not obvious for the new employees and external contributors. I propose to add a check that allows to add only unittest and luatest tests. Changing of existing tests, of course, should be allowed.

See also tarantool/test-run#304.

Footnotes

1. Multiline statements should be marked explicitly either by set delimiter test-run command or using a backslash at end of line. ↩