Running specmatic executable jar in gitlab pipeline but the tests are not running

[prateekkarmakar]

@harikrishnan83 , I have tried to run the specmatic executable from the gitlab CI pipeline

In the scripts I am using the following :
java -cp ${CI_PROJECT_DIR}/target/*:${CI_PROJECT_DIR}/target/$APP_NAME application.SpecmaticApplication test ${CI_PROJECT_DIR}/target/$TEST_CONTRACT_FILE --testBaseURL=http://localhost:8080

CI_PROJECT_DIR is the folder location of the build process like pwd in local.

I have included the dependencies required by specmatic executable jar in the target and loading them ( Please point me if anything is missing)

<plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-dependency-plugin</artifactId>
                <version>3.7.0</version>
                <executions>
                    <execution>
                        <id>copy-dependencies</id>
                        <phase>package</phase>
                        <goals>
                            <goal>copy</goal>
                        </goals>
                        <configuration>
                            <artifactItems>
                                <artifactItem>
                                    <groupId>io.specmatic</groupId>
                                    <artifactId>specmatic-executable</artifactId>
                                    <version>2.2.1</version>
                                    <outputDirectory>${project.build.directory}</outputDirectory>
                                </artifactItem>
                                <artifactItem>
                                    <groupId>org.jetbrains.kotlin</groupId>
                                    <artifactId>kotlin-stdlib</artifactId>
                                    <version>2.1.10</version>
                                    <outputDirectory>${project.build.directory}</outputDirectory>
                                </artifactItem>
                                <artifactItem>
                                    <groupId>io.specmatic</groupId>
                                    <artifactId>specmatic-core</artifactId>
                                    <version>${specmatic.version}</version>
                                    <outputDirectory>${project.build.directory}</outputDirectory>
                                </artifactItem>
                                <artifactItem>
                                    <groupId>io.specmatic</groupId>
                                    <artifactId>junit5-support</artifactId>
                                    <version>${specmatic.version}</version>
                                    <outputDirectory>${project.build.directory}</outputDirectory>
                                </artifactItem>
                                <artifactItem>
                                    <groupId>info.picocli</groupId>
                                    <artifactId>picocli</artifactId>
                                    <version>4.7.5</version>
                                    <outputDirectory>${project.build.directory}</outputDirectory>
                                </artifactItem>
                                <artifactItem>
                                    <groupId>org.springframework.boot</groupId>
                                    <artifactId>spring-boot</artifactId>
                                    <version>3.2.2</version>
                                    <outputDirectory>${project.build.directory}</outputDirectory>
                                </artifactItem>
                                <artifactItem>
                                    <groupId>org.springframework.boot</groupId>
                                    <artifactId>spring-boot-starter</artifactId>
                                    <version>3.2.2</version>
                                    <outputDirectory>${project.build.directory}</outputDirectory>
                                </artifactItem>
                                <artifactItem>
                                    <groupId>org.springframework</groupId>
                                    <artifactId>spring-context</artifactId>
                                    <version>6.1.3</version>
                                    <outputDirectory>${project.build.directory}</outputDirectory>
                                </artifactItem>
                                <artifactItem>
                                    <groupId>org.springframework</groupId>
                                    <artifactId>spring-core</artifactId>
                                    <version>6.1.3</version>
                                    <outputDirectory>${project.build.directory}</outputDirectory>
                                </artifactItem>
                                <artifactItem>
                                    <groupId>org.springframework</groupId>
                                    <artifactId>spring-beans</artifactId>
                                    <version>6.1.3</version>
                                    <outputDirectory>${project.build.directory}</outputDirectory>
                                </artifactItem>
                                <artifactItem>
                                    <groupId>commons-logging</groupId>
                                    <artifactId>commons-logging</artifactId>
                                    <version>1.3.4</version>
                                    <outputDirectory>${project.build.directory}</outputDirectory>
                                </artifactItem>
                                <artifactItem>
                                    <groupId>org.springframework</groupId>
                                    <artifactId>spring-aop</artifactId>
                                    <version>6.1.3</version>
                                    <outputDirectory>${project.build.directory}</outputDirectory>
                                </artifactItem>
                                <artifactItem>
                                    <groupId>org.springframework</groupId>
                                    <artifactId>spring-expression</artifactId>
                                    <version>6.1.3</version>
                                    <outputDirectory>${project.build.directory}</outputDirectory>
                                </artifactItem>
                            </artifactItems>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

Following is my run script for application under test and followed by the specmatic executable run

script:
  - |
    echo "1 Current Directory: $(pwd)"
    ARTIFACT_ID=$(mvn $MAVEN_CLI_OPTS $mvn_settings_opt $java_proxy_args help:evaluate -Dexpression=project.artifactId -q -DforceStdout | tail -n 1)
    VERSION=$(mvn $MAVEN_CLI_OPTS $mvn_settings_opt $java_proxy_args help:evaluate -Dexpression=project.version -q -DforceStdout | tail -n 1)
    APP_NAME="${ARTIFACT_ID}-${VERSION}.jar"
    echo "Application Name: $APP_NAME"
    curl -o ${CI_PROJECT_DIR}/target/$TEST_CONTRACT_FILE $CENTRAL_CONTRACT_REPO_URL
    echo "1 Current Directory: $TEST_CONTRACT_FILE"
    echo "2 CI_PROJECT_DIR : ${CI_PROJECT_DIR}"
    **mv ${CI_PROJECT_DIR}/target/specmatic-executable-2.2.1.jar ${CI_PROJECT_DIR}/target/specmatic.jar**
    # Start the application.jar
    **nohup java -jar ${CI_PROJECT_DIR}/target/$APP_NAME &**
    # Wait for the application to start
    **sleep 30**
    # Run the specmatic.jar command
    **java -cp ${CI_PROJECT_DIR}/target/*:${CI_PROJECT_DIR}/target/$APP_NAME application.SpecmaticApplication test ${CI_PROJECT_DIR}/target/$TEST_CONTRACT_FILE --testBaseURL=http://localhost:8080**
     echo "COMPLETE"

In the logs I find the following

....................
2025-02-03T11:15:44.341Z INFO 225 --- [customer] [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 14 endpoint(s) beneath base path '/actuator'
2025-02-03T11:15:44.417Z INFO 225 --- [customer] [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port 8080 (http) with context path ''
2025-02-03T11:15:44.428Z INFO 225 --- [customer] [ main] c.testing.customer.CustomerApplication : Started CustomerApplication in 5.228 seconds (process running for 5.7)
Specmatic Version: 2.2.1
[2025-02-03 11:16:09.163] - 266 INFO [main] --- application.SpecmaticApplication$Companion: Starting SpecmaticApplication.Companion using Java 17.0.1 with PID 266 (/tmp/gitlab-build/testing/git/GQGIIG/specmatic/specmatic-customer-java11/target/specmatic.jar started by ? in /tmp/gitlab-build/testing/git/GQGIIG/specmatic/specmatic-customer-java11)
[2025-02-03 11:16:09.183] - 266 INFO [main] --- application.SpecmaticApplication$Companion: No active profile set, falling back to 1 default profile: "default"
[2025-02-03 11:16:09.277] - 266 INFO [main] --- application.SpecmaticApplication$Companion: Started SpecmaticApplication.Companion in 0.357 seconds (process running for 0.547)
COMPLETE
Saving cache for successful job
00:01
Creating cache main-maven-protected...
./.m2/repository: found 4958 matching artifact files and directories
Archive is up to date!
Created cache
Cleaning up project directory and file based variables
00:00
Job succeeded
............................................

The Tests are not logged and the process stops. In local the process works fine

C:\PRATEEK\SPECMATIC_CUSTOMER_JAVA11>java -jar specmatic.jar test C:\PRATEEK\SPECMATIC_CUSTOMER_JAVA11\specmatic-customer\local-spec\customer.yaml --testBaseURL=http://localhost:8080
Specmatic Version: 2.2.1

Could not find the Specmatic configuration at path C:\PRATEEK\SPECMATIC_CUSTOMER_JAVA11\specmatic.yaml
API Specification Summary: C:\PRATEEK\SPECMATIC_CUSTOMER_JAVA11\specmatic-customer\local-spec\customer.yaml
OpenAPI Version: 3.0.1
API Paths: 2, API Operations: 3
Schema components: 3, Security Schemes: none

Endpoints API not found, cannot calculate actual coverage

Scenario: POST /customers -> 200 | EX:CUSTOMER_ADDRESS has FAILED
Reason: Testing scenario "POST /customers. Response: OK"
API: POST /customers -> 200

      Connection refused: no further information

Scenario: POST /customers -> 200 | EX:CUSTOMER has FAILED
Reason: Testing scenario "POST /customers. Response: OK"
API: POST /customers -> 200

      Connection refused: no further information

Scenario: GET /customers -> 200 | EX:SUCCESS has FAILED
Reason: Testing scenario "GET /customers. Response: OK"
API: GET /customers -> 200

      Connection refused: no further information

Scenario: GET /customers/(customerId:number) -> 200 | EX:GET_NON_VERIFIED_CUSTOMER has FAILED
Reason: Testing scenario "GET /customers/(customerId:number). Response: OK"
API: GET /customers/(customerId:number) -> 200

      Connection refused: no further information

My Question is my command wrong? If its correct what changes I need to make to make the executable running in pipeline?

I am trying to run the executable in the pipeline to test the scenario where code is of java 11 and it cannot run specmatic tests during build. REF>> #1493

I had to add the jars in classpath and the main class name in command as without them I was always getting error like

no main manifest attribute, in /tmp/gitlab-build/testing/git/GQGIIG/specmatic/specmatic-customer-java11/target/specmatic.jar

OR

Exception in thread "main" java.lang.NoClassDefFoundError: io/specmatic/core/utilities/UncaughtExceptionHandler
at application.SpecmaticApplication$Companion.main(SpecmaticApplication.kt:21)
at application.SpecmaticApplication.main(SpecmaticApplication.kt)
Caused by: java.lang.ClassNotFoundException: io.specmatic.core.utilities.UncaughtExceptionHandler
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
... 2 more

Thanks for your guidance in advance.