ScoverageAggregate for different test

[aesteve]

Hi and sorry to create an issue, since it's more of a question than a proper issue, but how do you aggregate coverage when you have different test tasks?

I have a multi-project build.
Say I have two test tasks: test and integrationTest
Then I have (as you said): reportTestScoverage and reportIntegrationTestScoverage => perfect.
I tried aggregateScoverage: it's running both the test and integrationTest tasks, perfect.
But I'd need to have two different aggregateScoverage tasks, one for each test task.

I tried first:

        create<org.scoverage.ScoverageAggregate>("testCoverage") {
            dependsOn("reportTestScoverage")
        }

But it failed saying many properties were missing.

Thus I tried:

        create<org.scoverage.ScoverageAggregate>("testCoverage") {
            dependsOn("reportTestScoverage")
            coverageOutputCobertura.set(true)
            coverageOutputHTML.set(true)
            coverageOutputXML.set(true)
            deleteReportsOnAggregation.set(false)
            coverageDebug.set(false)
            reportDir.set(rootProject.buildDir.resolve("reports"))
        }

But this failed because ScoverageAggregate.runner is null.

So I finally got it working through:

        create<org.scoverage.ScoverageAggregate>("testCoverage") {
            dependsOn("reportTestScoverage")
            coverageOutputCobertura.set(true)
            coverageOutputHTML.set(true)
            coverageOutputXML.set(true)
            deleteReportsOnAggregation.set(false)
            coverageDebug.set(false)
            reportDir.set(rootProject.buildDir.resolve("reports"))
            runner = ScoverageRunner(project.configurations.scoverage.get())
        }

But it doesn't work, it only creates scoverage for the last subProject it builded.

Do you have any pointer? Thank you!

[maiflai]

I'm not sure this is going to work in the ideal fashion, mainly because of the way that scoverage currently works under the covers.

Coverage of a particular class is measured and written to a location fixed during compilation.

I think this means that you won't be able to easily separate coverage obtained under unit or integration tests in a single build.

You might try using -x test on the command line, which should then only run the integrationTest tasks? You would then need to run another build skipping the test tasks, possibly into a different buildDir if you want to retain the results.

Stu.

[aesteve]

Ok so standard aggregateScoverage but with -x option, and two different reports.

I can live with that I think. Mostly, test should have maximum coverage except for the environment-dependent stuff.
Then, integrationTest should just check that given the right set of environment variables, it "boots", and "does some stuff".

So I guess I can merge both "manually" by asserting integration have covered the stuff we need it to cover (env variables and App stuff).

Thanks for your help!

[eyalroth]

You might try using -x test on the command line, which should then only run the integrationTest tasks? You would then need to run another build skipping the test tasks, possibly into a different buildDir if you want to retain the results.

I also think that's the way to go, but with a slight adjustment. Instead of omitting the test task, try omitting the specific report task instead:

gradle aggregateScoverage -x reportTestScoverage
or
gradle aggregateScoverage -x reportIntegrationTestScoverage

Make sure to clean between the two, since the aggregation task will not be able to tell which report files were generated on the first build and which on the second.

[atais]

I used

gradle aggregateScoverage -x reportTestScoverage

and it still fires integrationTest task.

My test config:

apply plugin: "org.scoverage"
apply plugin: "com.github.maiflai.scalatest"

test {
    tags {
        exclude "org.scalatest.tags.Slow"
    }
}

task integrationTest(type: Test) {
    tags {
        include "org.scalatest.tags.Slow"
    }
}