The cross tests test for the interchangeability of the javabite and the fuc modules.
The cross tests can be found in
crosstesting/crosstest.
They are parameterized over the Cartesian product of all 32 module combinations,
so every example programme is tested against every possible combination of javabite
and fuc modules. The cross tests compile the example programme and assert, that the
the compilation either produces expected errors or compiles through, producing
some kind of target code. The resulting target code is then executed and the
result of the execution is checked against the expected output
(print statements) and exit code (return statement). The runtime tests
hereby provide limited correctness testing (no proof of correctness of course!).
The example programmes are centralized in
ExampleProgs.java.
The example programmes comprise of the 15 examples from
common/examples
(loaded from there) plus 9 additional test programmes. In conjunction with the 32
module combinations this results in 768 test cases.
The main test logic is centralized in AbstractCrosstest.java,
while all the example programmes are implemented as test classes inheriting from AbstractCrosstest.java.
Every commit pushed to the remote master branch is tested with Travis CI. Before running the tests, LLVM is installed, allowing the execution of the the runtime tests. The current build status of the master branch is displayed at the top. The testing can be seen in action an the Travis CI status page for crosstesting.
Use recursive cloning, to get the submodules with the repo:
git clone [email protected]:swp-uebersetzerbau-ss13/crosstesting.git --recursive
If already cloned, use git submodule update --init --recursive to get the
submodules to update to the commited version.
gradle :jbJars to build the javabite jars in javabite/bin [1]
gradle :fucJars to build the fuc jars in fuc/code/dist [1]
gradle :crosstest:test [1]
Nicely formatted results of the crosstests can be found here. This results page is updated manually, it therefore does not necessarily reflect the current stage of the repository.
Always up-to-date results can be found on the Travis CI Project Page.
[1] If gradle is not installed, ./gradlew <task> can be used instead