Task: Set up Test Coverage Report

[NightOwl888]

While adding more tests is probably not an activity we will be focusing on outside of custom components and untested APIs that had significant refactoring, having some insight as to which APIs carry more risk of change than others would be useful. Also, knowing the percentage of test coverage would help us to convey how good or bad the coverage is for a particular component.

There are 2 parts to this.

• Workflow Integration
• Mailing List Notification

Workflow Integration

Thanks to @nikcio, we now know that running the test coverage report requires about 3-4 hours of server time, perhaps more. If we use GitHub Actions, Apache doesn't put a time limit on agents.

Also, thanks to @nikcio, we know that SonarCloud can do some of the heavy lifting as far as UI integration with the report. So, I think adding the test coverage to the sonar.yml GitHub Workflow on a schedule (perhaps daily, perhaps less) is probably our best way of getting these reports.

We should probably divide up the scan into 2 triggers (assuming SonarCloud can deal with this):

1. A push to master trigger that launches the SonarCloud scan only
2. A scheduled event trigger that runs both the test coverage and the SonarCloud scan

Perhaps there is some way to cache the last code coverage report so the push scans can just "inherit" the coverage report from the full scan.

GitHub Actions allows tasks/jobs to run conditionally based on the trigger that fired them, which would allow us to conditionally run the code coverage. See the schedule trigger documentation.

Notification

We have already set up a build mailing list to send notifications for nightly and release builds (and whether they passed/failed). We can send an email to that mailing list when SonarCloud finishes its task to notify us that the report is ready for viewing (only when running on a schedule - pushes would cause a lot of noise).