Customizing coverage result client?

[Antoine-Lassauzay]

Hi there.

First of all thank you for munit and mcoverage ; I'm really glad that you guys brought CI tools for HaXe.

I have a question though ; I don't really understand how to add/customize the way coverage results output. I can see the details in the AS3 runner, but report.txt file only provides projet level information.

Also, I would like to try the EMMAPrintClient that you provide ; how to you add it (I tried MCoverage.getLogger().addClient but no luck). Is it functional?

Eventually I might need to write a Cobertura client, so clarifying how you can swap implementations in this particular area of the test runner settings would be nice.

Thanks

[misprintt]

Hi,

Apologies - it is a little confusing.

The short answer is that anything longer than a project summary report can potentially cause munit's server to hang. This is due to a bug in the nekotools server with sending multiple POST requests with large amounts of data (it never clears the buffer between requests). I believe the bug has been fixed on neko svn, but there hasn't been a proper update to Neko in a long time (although Nicolas mentioned a month or so ago that a public release is on the cards soonish)

We originally wrote the EMMAPrintClient (and Munit's JUnitReportClient) with the intension that these could be saved out by munit's CLI. While they worked in simple test apps, in real world apps (with plenty of tests and coverage results) they consistently hung the CLI.

We switched to the simple summary report as a fallback for CI integration that was small enough to never hit the POST limit, but still enable basic validation of tests/coverage.

Munit2 (which we work on and off where we can) has switched to Node rather than NekoTools server (and websockets over POST) to work around this issue, however we dont have an estimate release window just yet.

I have also began work on a dedicated CLI for MCover, that will convert a raw serialised coverage result data set into different report formats. In the longer term this will be a more efficient approach, as MUnit2 will only need to handle a single coverage format, and users can convert this into xml/html reports on completion via haxelib run mcover report ...

In the meantime you have a few options.

1. For system targets (c++, neko, etc) you can write a custom report out to a file manually in the TestMain once the tests have completed. Unfortunately for JS and AS3 targets this wont work.
2. Generate the full report once the test runner has completed and then send it to a custom service (e.g. a simple node server) to save the report out to the file system.

If you end up writing a CoberturaPrintClient we'd love to include it in a release.

One final option may be to write a cobertura friendly summary report format for the MUnit CLI (if Cobertura supports a summary formats).

You can use the TeamCity variant as an example (haxelib run munit report teamcity). We use this to integrate summary results/coverage with our CI environment. Source is here: http://github.com/massiveinteractive/MassiveUnit/tree/master/tool/src/massive/munit/report.

Hope that helps a little...

Dom