Drill makes extensive use of JUnit and other libraries for testing. This page provides pointers to the information you need to work with Drill tests. We don't repeat that information; you will want to follow the links and read the original material to get a complete understanding of the libraries that Drill uses.
This is a list of old Drill testing machinery that we have cleaner machinery for now.
- BaseTestQuery: Deprecated, use ClusterTest instead.
This is a list of old Drill testing machinery for which there is no other alternative at the moment.
These are all the latest Drill testing Techniques that you should use. If you find one of these approaches insufficient for your use, please enhance them or file a Jira so that they an be improved.
- RowSet Framework
- Cluster Fixture Framework
- Operator Fixture Framework
- ClusterTest
- Single Operator Unit Test
- Instantiating Components
- Generated Code
- MiniPlanUnitTestBase
When testing an operator or a specific internal mechanism in Drill you should use unit tests and should avoid depending on integration tests. This approach means that you will have to design your implementation to be decoupled from the rest of Drill. While this seems daunting, in reality it doesn't take much time and you will spend much less time debugging and stressing about customer bug fixes.
Drill provides several tools to facilitate unit testing. See:
- RowSet Framework
- Operator Fixture Framework
- Single Operator Unit Test
- Instantiating Components
- Generated Code
The following are some examples of operators which have leveraged these techniques for unit testing:
After the unit tests are implemented and passing, use integration tests to ensure the whole system works. The standard tools for accomplishing integration testing are:
The general rule is: test as close to your code as possible, refactoring and breaking dependencies where needed to accomplish this. On the other hand, if the test is more of a SQL or planner-level concept, then testing at the query level might be fine.
Currently Drill uses Travis to run smoke tests for every PR and commit. All of Drill's unit tests cannot be run on Travis because Drill's tests take longer to run than the
maximum allowed container time for the free tier of Travis. In order to decide which tests are run on Travis and which tests are not, tests are categorized using JUnit's
@Category annotation. Currently the following categories are excluded from Travis:
- SlowTest: Tests that are slow.
- UnlikelyTest: Tests that cover parts of the code that are rarely or never touched.
- SecurityTest: Corner case tests for security features.
To mark a test with a category you can do the following:
@Category(SlowTest.class)
public class MyTest {
// Testing code
}
To mark a test with multiple categories you can do the following:
@Category({SlowTest.class, SecurityTest.class})
public class MyTest {
// Testing code
}
Drill tests run in parallel. The model for parallel execution is to divide test classes between multiple forked test processes. Each test process then runs the test classes assigned to it sequentially.
There are a couple knobs you can turn to make tests run faster on your machine.
- Maven Build Threads:
-T <num> - Sure Fire Fork Count:
-DforkCount=<num> - Test Categories
Maven allows you to use multiple threads to compile sub modules. Also when running tests each build
thread forks its own surefire process, so the tests for different submodules are run in parallel. In order
to leverage this use the -T flag. By default this option is effectively 1, so there is only one build thread by default.
Ex. In order to run the build using two maven threads use the following command.
mvn -T 2 clean install
To run tests within a submodule in parallel you can use the -DforkCount option. By default this 2, so two surefire processes are forked for each build thread.
Ex. Run 4 test processes in parallel
mvn clean install -DforkCount=4
Note: The -DforkCount option interacts with -T. When used together each build thread (-T) gets
-DforkCount test processes.
You can leverage categories to run subsets of the tests. This is useful if you need to test something like a storage plugin on a remote jenkins server. See the java docs here for examples. For a list of all the availabe categories go here.
It is often helpful to know the set of properties that Maven defines and are available for use in the POM file.
Drill uses the Maven Surefire plugin to run tests. The root POM contains significant test configuration that you may or may not want to include when running tests in a debugger:
<configuration>
<argLine>-Xms512m -Xmx3g -Ddrill.exec.http.enabled=false
-Ddrill.exec.sys.store.provider.local.write=false
-Dorg.apache.drill.exec.server.Drillbit.system_options=\
"org.apache.drill.exec.compile.ClassTransformer.scalar_replacement=on"
-Ddrill.catastrophic_to_standard_out=true
-XX:MaxPermSize=512M -XX:MaxDirectMemorySize=3072M
-Djava.net.preferIPv4Stack=true
-Djava.awt.headless=true
-XX:+CMSClassUnloadingEnabled -ea</argLine>
<forkCount>${forkCount}</forkCount>
<reuseForks>true</reuseForks>
<additionalClasspathElements>
<additionalClasspathElement>./exec/jdbc/src/test/resources/storage-plugins.json</additionalClasspathElement>
</additionalClasspathElements>
You can run individual tests as follows. First, cd into the project that contains the test. Then, run the test. (You'll get an error if you try to run the test from the root Drill directory.)
cd /path/to/drill/project
mvn surefire:test -Dtest=TestCircle
Drill code makes sparse use of Java exceptions, preferring the Guava equivalents. One area that assertions (-ea option) does affect is performance: queries run about 10x slower with assertions enabled due to allocator logging. When doing any performance-related work, ensure that assertions are not enabled.
class DrillBuf ...
public boolean release(int decrement) {
...
if (BaseAllocator.DEBUG) {
historicalLog.recordEvent("release(%d). original value: %d", decrement, refCnt + decrement);
}
TBA