-
Notifications
You must be signed in to change notification settings - Fork 733
Reproducing Test Failures Locally
-
Ensure test machine is set up with test prereqs.
-
Build or download/unpack the SDK to /someLocation
-
git clone https://github.com/AdoptOpenJDK/openjdk-tests.git (to /testLocation)
-
cd openjdk-tests
-
./get.sh -t /testLocation/openjdk-tests -p < somePlatform > -i < someImpl >
-
cd TestConfig
-
export environment variables suitable for the SDK under test
-
make -f run_configure.mk // generates test targets applicable to the platform, implementation and version
-
make compile // fetches test material and compiles it, based on build.xml files in the test directories
-
make _< someTestTarget > // executes the test target (can be test group, level, level.group or specific test). i.e., openjdk (all tests in openjdk group), sanity.functional (all functional tests labelled at sanity level), extended.system (all system tests labelled at extended level), jdk_math (the specific jdk_math target defined as part of openjdk group), MauveMultiThreadLoadTest_0 (the first variation of the specific system test called MauveMultiThreadLoadTest), etc
Below is a list of common scenarios. As you can see, the approach to running the different groups of tests is very similar. With minimal changes to environment variables and target names, a great variety of test scenarios can be run. These instructions can be used for running any group of tests on any JDK version, any JDK implementation, and any platform/spec. For more details, please see the OpenJ9 Test Documentation.
There are a number of degrees of freedom by which you can generate, compile and execute test targets. Some variants are:
- test group [functional|system|openjdk|external|perf]
- level [sanity|extended|special]
- jdk version - [8|9|10|11]
- jdk impl - [openj9|hotspot|sap|ibm]
- platform - [x64_linux|x64_mac|s390x_linux|ppc64le_linux|aarch64_linux|ppc64_aix]
- spec - [linux_x86-64_cmprssptrs|linux_x86-64| etc] - as listed in buildspecs
Test group | JDK version | JDK impl | Platform/Spec | get.sh options | Env variables |
---|---|---|---|---|---|
functional | 8 | openj9 | x64_linux/linux_x86-64_cmprssptrs | -p x64_linux -i openj9 | export JAVA_BIN=/someLocation/jre/bin export SPEC=linux_x86-64_cmprssptrs export JDK_VERSION=8 export JDK_IMPL=openj9 export BUILD_LIST=functional |
functional | 10 | openj9 | x64_linux/linux_x86-64_cmprssptrs | -p x64_linux -i openj9 | export JAVA_BIN=/someLocation/bin export SPEC=linux_x86-64_cmprssptrs export JDK_VERSION=10 export JDK_IMPL=openj9 export BUILD_LIST=functional |
system | 10 | openj9 | x64_linux/linux_x86-64_cmprssptrs | -p x64_linux -i openj9 | export JAVA_BIN=/someLocation/bin export SPEC=linux_x86-64_cmprssptrs export JDK_VERSION=10 export JDK_IMPL=openj9 export BUILD_LIST=systemtest |
openjdk | 11 | hotspot | x64_mac/macos_x86-64 | -p x64_mac -i hotspot | export JAVA_BIN=/someLocation/bin export SPEC=macos_x86-64 export JDK_VERSION=11 export JDK_IMPL=hotspot export BUILD_LIST=openjdk_regression |
openjdk | 11 | openj9 | s390x_linux/linux_390-64_cmprssptrs | -p s390x_linux -i openj9 | export JAVA_BIN=/someLocation/bin export SPEC=linux_390-64_cmprssptrs export JDK_VERSION=11 export JDK_IMPL=openj9 export BUILD_LIST=openjdk_regression export JDK_CUSTOM_TARGET=java/lang/Class/GetModuleTest.java |
external | 8 | openj9 | x86_linux/linux_x86-64_cmprssptrs | -p x64_linux -i openj9 | export JAVA_BIN=/someLocation/jre/bin export SPEC=linux_x86-64_cmprssptrs export JDK_VERSION=8 export JDK_IMPL=openj9 export BUILD_LIST=thirdparty_containers |
perf | 11 | openj9 | x86_linux/linux_x86-64_cmprssptrs | -p x64_linux -i openj9 | export JAVA_BIN=/someLocation/bin export SPEC=linux_x86-64_cmprssptrs export JDK_VERSION=11 export JDK_IMPL=openj9 export BUILD_LIST=performance |
- Build or unpack the jdk8_x86-64_linux_openj9 SDK to /someLocation
- git clone https://github.com/AdoptOpenJDK/openjdk-tests.git (to /testLocation)
- cd openjdk-tests
- ./get.sh -t /testLocation/openjdk-tests -p x64_linux -i openj9
- cd TestConfig
- export environment variables matching the SDK under test (
export JAVA_BIN=/someLocation/jre/bin export SPEC=linux_x86-64_cmprssptrs export JDK_VERSION=8 export JDK_IMPL=openj9 export BUILD_LIST=functional
) - for jdk8, JAVA_BIN -> jre/bin - make -f run_configure.mk
- make compile
- make _extended.functional
- Build or unpack the jdk10_x86-64_linux_openj9 SDK to /someLocation
- git clone https://github.com/AdoptOpenJDK/openjdk-tests.git (to /testLocation)
- cd openjdk-tests
- ./get.sh -t /testLocation/openjdk-tests -p x64_linux -i openj9
- cd TestConfig
- export environment variables matching the SDK under test (
export JAVA_BIN=/someLocation/bin export SPEC=linux_x86-64_cmprssptrs export JDK_VERSION=10 export JDK_IMPL=openj9 export BUILD_LIST=functional
) - make -f run_configure.mk
- make compile
- make -C ../functional/JIT_Test -f autoGen.mk _sanity
- Build or unpack the jdk10_x86-64_linux_openj9 SDK to /someLocation
- git clone https://github.com/AdoptOpenJDK/openjdk-tests.git (to /testLocation)
- cd openjdk-tests
- ./get.sh -t /testLocation/openjdk-tests -p x64_linux -i openj9
- cd TestConfig
- export environment variables matching the SDK under test (
export JAVA_BIN=/someLocation/bin export SPEC=linux_x86-64_cmprssptrs export JDK_VERSION=10 export JDK_IMPL=openj9 export BUILD_LIST=systemtest
) - make -f run_configure.mk
- make compile
- make _sanity.system
- Build or unpack the jdk11_x64_macos_hotspot SDK to /someLocation
- git clone https://github.com/AdoptOpenJDK/openjdk-tests.git (to /testLocation)
- cd openjdk-tests
- ./get.sh -t /testLocation/openjdk-tests -p x64_mac -i hotspot
- cd TestConfig
- export environment variables matching the SDK under test (
export JAVA_BIN=/someLocation/bin export SPEC=macos_x86-64 export JDK_VERSION=11 export JDK_IMPL=hotspot export BUILD_LIST=openjdk_regression
) - make -f run_configure.mk
- make compile
- make _sanity.openjdk
For this example, let's run a currently excluded test, java/lang/Class/GetModuleTest.java found in exclude file
- Build or unpack the jdk11-linux-s390x-openj9 SDK to /someLocation
- git clone https://github.com/AdoptOpenJDK/openjdk-tests.git (to /testLocation)
- cd openjdk-tests
- ./get.sh -t /testLocation/openjdk-tests -p s390x_linux -i openj9
- cd TestConfig
- export environment variables matching the SDK under test
(
export JAVA_BIN=/someLocation/bin export SPEC=linux_390-64_cmprssptrs export JDK_VERSION=11 export JDK_IMPL=hotspot export BUILD_LIST=openjdk_regression export JDK_CUSTOM_TARGET=java/lang/Class/GetModuleTest.java
) - make -f run_configure.mk
- make compile
- make _jdk_custom
- Build or unpack the jdk8-x86-64_linux_openj9 SDK to /someLocation
- git clone https://github.com/AdoptOpenJDK/openjdk-tests.git (to /testLocation)
- cd openjdk-tests
- ./get.sh -t /testLocation/openjdk-tests -p x64_linux -i openj9
- cd TestConfig
- export environment variables matching the SDK under test (
export JAVA_BIN=/someLocation/jre/bin export SPEC=linux_x86-64_cmprssptrs export JDK_VERSION=8 export JDK_IMPL=openj9 export BUILD_LIST=thirdparty_containers
) - for jdk8, JAVA_BIN -> jre/bin - make -f run_configure.mk
- make compile
- make _scala_test
- Build or unpack the jdk11-x86-64_linux_openj9 SDK to /someLocation
- git clone https://github.com/AdoptOpenJDK/openjdk-tests.git (to /testLocation)
- cd openjdk-tests
- ./get.sh -t /testLocation/openjdk-tests -p x64_linux -i openj9
- cd TestConfig
- export environment variables matching the SDK under test (
export JAVA_BIN=/someLocation/bin export SPEC=linux_x86-64_cmprssptrs export JDK_VERSION=11 export JDK_IMPL=openj9 export BUILD_LIST=performance
) - make -f run_configure.mk
- make compile
- make _IdleMicrobenchmark_J9