[PoC] Adding Cypress tests code coverage

[CarlosCortizasCT]

Summary

Here is a PoC where we try to setup a code coverage report out of the e2e tests we run on the Playground application.

Description

We basically need to:

• Make sure we instrument our application code
• Build the application with the instrumented code
• Configure Cypress to read the instrumentation data
• Run Cypress agains the instrumented built application

When we run the instrumented application we will see a window.__coverage__ object in the browser which contains data and helpers which Cypress e2e support plugin will use in order to gather information.
Cypress will generate a low level report in the <repo_root>/.nyc_output/out.json file and we can use it to generate a human readable report.

These would be the steps:

1. Make sure we have built the repo packages: pnpm build
2. Configure the Playground application to be built with webpack (ENABLE_EXPERIMENTAL_VITE_BUNDLER env variable set to 'false' in the /playground/.env file)
3. Build the application with instrumentation (in this branch we always instrument code during builds): pnpm playground:build
4. Run the instrumented built application locally: pnpm playground:start:prod:local
5. Make sure you have the Cypress env variables configured (take a look at the /cypress/.env.template file)
6. Run Cypress locally: I just run one test from the Cypress UI
7. npx cypress open
8. Select Playground -> application-shell.cy
9. Convert the output coverage file into a human readable one by running npx nyc report (from the root folder of the repo)

This is meant to play with the configuration and make sure we can apply it to every MC application and get a report we can trust.

Important Currently the coverage report does not include info for the page components of playground. It seems it just reads the index.js file of those components which are just a lazy export of the actual ones.
We need to investigate and have this sorted out before we can take this PoC as successful.
Also, the current implementation has been only tested when building the application with webpack. We need to test it with vite as well.

⚠️ THIS PR IS NOT MEANT TO BE MERGED ⚠️

[changeset-bot]

⚠️ No Changeset found

Latest commit: c0d841d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
mc-app-kit-playground	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 31, 2024 1:04pm
merchant-center-application-kit-components-playground	❌ Failed (Inspect)			Oct 31, 2024 1:04pm

[ismaelocaramelo]

Steps to Generate a Unified Coverage Report

1. Instrument Code for Coverage
   Ensure that the codebase is properly instrumented to collect coverage data. This is a prerequisite for both Jest and Cypress tests to generate accurate coverage reports.

2. Execute Cypress Tests
   Run the Cypress test suite to capture end-to-end test coverage. Upon successful completion, verify the presence of a JSON file with a valid coverage object in the .nyc_output directory. The file should resemble the following structure:

   {
     "path": "src/index.js",
     "statementMap": {
       "0": { "start": { "line": 1, "column": 0 }, "end": { "line": 1, "column": 12 } }
     },
     "fnMap": {},
     "branchMap": {},
     "s": { "0": 1 }
   }

3. Execute Jest Tests
   Run the Jest test suite to capture unit and integration test coverage. This process should generate a coverage-final.json file within the jest-coverage directory.

4. Copy Coverage Files
   Utilize the command pnpm copy:reports to transfer the coverage files from their respective directories to a centralized reports folder.

5. Combine Coverage Reports
   Execute pnpm combine:reports to merge the Cypress and Jest coverage data into a single comprehensive report.

6. Generate the Final Coverage Report
   Run pnpm report:combined to produce the final unified coverage report. This command will generate an HTML report that consolidates the coverage data from both testing frameworks.

7. Review the Coverage Report
   Open the index.html file located in the coverage directory to review the detailed coverage report. This report provides insights into the overall test coverage across the codebase.

---