feat: implement e2e tests workflow #227

huynguyen-hl · 2025-02-28T11:04:41Z

What type of PR is this? (check all applicable)

Description

This PR sets up Cypress E2E tests to run in both the Build and Test and Release workflows. The tests will be triggered whenever:

A commit is pushed to the main branch (release).
A PR is raised into the next branch.

Steps added to run E2E tests:

Start E2E Docker Compose.
Ensure IDR Service Readiness and Health (call the IDR health check API to ensure it’s ready to receive test requests).
Run E2E tests.
Stop Docker Compose.

In this PR, I’ve updated the resetData function to use the MinIO client library for deleting data in the identity-resolver-service-object-store service instead of the fs library for file system interactions. Additionally, I’ve renamed the function to clearObjectStore for better clarity. This removes the need to manually delete files or folders that are bind-mounted by the identity-resolver-service-object-store Docker Compose service, which would otherwise lead to a file system permission issue. Since the GitHub Actions environment runs under the runner user while the bind-mounted files and folders are owned by root, direct deletion is not possible.

Cypress E2E tests run successfully in a local environment because we use the same user (which has the highest privileges). However, if a different user were used locally, the same issue would occur.

Mobile & Desktop Screenshots/Recordings

Testing E2E pipeline in Github personal account:

Added tests?

👍 yes
🙅 no, because they aren't needed
🙋 no, because I need help

Added to documentation?

📖 Mock App docs site
📜 README.md
📕 storybook
🙅 no documentation needed

…kflows

github-actions · 2025-02-28T11:09:08Z

Code Coverage Report

Lines	Statements	Branches	Functions
	81.79% (3571/4366)	69.1% (859/1243)	73.54% (417/567)

Title	Statements	Branches	Functions
All packages	81.79% (3571/4366)	69.1% (859/1243)	73.54% (417/567)
Components	84.03% (516/614)	69.36% (120/173)	77.23% (95/123)
Mock app	71.89% (371/516)	53.23% (74/139)	68.14% (77/113)
Services	79.29% (1038/1309)	64.88% (255/393)	75.71% (106/140)
UNTP test suite	85.75% (1090/1271)	81.35% (144/177)	67.21% (41/61)
VC test suite	100% (20/20)	100% (2/2)	100% (4/4)
UNTP Playground	84.92% (569/670)	73.68% (266/361)	75.38% (98/130)

absoludity · 2025-03-03T00:16:04Z

Hi @huynguyen-hl . This PR is in a Draft status (and has conflicts), and has some notes about things that aren't finished, so it's not clear if you're wanting a review yet?

Commented out cy.task('resetData') due to a file system permission conflict in the GitHub Actions environment. The minio_data folder is created by a Docker bind mount with root privileges, but the GitHub workflow runs under the runner user, which lacks permission to delete files inside minio_data (This is a temporary workaround until a better solution is implemented).

Can we just sudo to delete the files from within the runner? See https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#administrative-privileges

…idation

huynguyen-hl · 2025-03-03T03:48:23Z

Thanks @absoludity for the early feedback. I tried using sudo, but it still resulted in permission errors. I also used the mxschmitt/action-tmate@v3 action to SSH into the GitHub Actions server for debugging and discovered that the root cause was the difference in folder ownership.

To address this, I took a different approach by using the MinIO client library to delete data from the identity-resolver-service-object-store service. This method works as intended without any permission issues.

huynguyen-hl · 2025-03-03T03:52:30Z

I noticed that some test cases in both untp-playground/report-generation.cy.ts and untp-playground/vcdm-validation.cy.ts are using unavailable files, such as:

cypress/fixtures/credentials-e2e/valid-v2-enveloped-dpp.json
cypress/fixtures/credentials-e2e/invalid-schema-v2-enveloped-dpp.json
...

This is why the E2E tests are failing. I believe we'll need a separate ticket to address this issue.

ashleythedeveloper · 2025-03-03T04:52:46Z

I noticed that some test cases in both untp-playground/report-generation.cy.ts and untp-playground/vcdm-validation.cy.ts are using unavailable files, such as:

cypress/fixtures/credentials-e2e/valid-v2-enveloped-dpp.json

cypress/fixtures/credentials-e2e/invalid-schema-v2-enveloped-dpp.json

...

This is why the E2E tests are failing. I believe we'll need a separate ticket to address this issue.

@huynguyen-hl, this is the fix I was referring to in our last standup. I've raised PR #229 to address this issue. I suggest to merge PR #229 first.

ashleythedeveloper · 2025-03-03T05:26:44Z

.github/workflows/build_test.yml

+      - name: Start E2E docker compose
+        run: SEEDING=true docker compose -f docker-compose.e2e.yml up -d
+
+      - name: Ensure IDR Service Readiness and Health


Hi @huynguyen-hl, is there a reason why we aren't using the built-in docker-compose health check functionality?

ashleythedeveloper · 2025-03-03T05:27:04Z

.github/workflows/release.yml

+      - name: Start E2E docker compose
+        run: SEEDING=true docker compose -f docker-compose.e2e.yml up -d
+
+      - name: Ensure IDR Service Readiness and Health


Same as above.

ashleythedeveloper · 2025-03-03T05:29:01Z

.github/workflows/build_test.yml

@@ -51,6 +51,52 @@ jobs:
            VC test suite, ./packages/vc-test-suite/coverage/coverage-summary.json
            UNTP Playground, ./packages/untp-playground/coverage/coverage-summary.json

+      - name: Start E2E docker compose
+        run: SEEDING=true docker compose -f docker-compose.e2e.yml up -d


Will GitHub cache the docker image between each run? If so, we need to build the image so the new functionality is present in the containers.

ashleythedeveloper · 2025-03-03T05:29:16Z

.github/workflows/release.yml

@@ -41,6 +41,52 @@ jobs:
      - name: Run tests
        run: yarn test

+      - name: Start E2E docker compose
+        run: SEEDING=true docker compose -f docker-compose.e2e.yml up -d


Same as above.

ashleythedeveloper · 2025-03-03T05:36:46Z

e2e/cypress.config.ts


 const execPromise = util.promisify(exec);
 export default defineConfig({
+  env: {
+    idrBucketName: process.env.IDR_BUCKET_NAME || 'idr-bucket-1',


I'm not sure why we are diverging from the existing environment variable names for the IDR services.

huynguyen-hl added 4 commits February 28, 2025 16:44

chore: update E2E testing workflow and disable specific test cases

a90b705

feat: add E2E testing setup in Release's test_and_build pipeline job

ccb73f3

chore: remove workflow_dispatch trigger from build_test.yml

00e0c8b

refactor: improve IDR service readiness and health check logic in wor…

e0f0c3e

…kflows

huynguyen-hl requested review from absoludity, ldhyen99 and ashleythedeveloper February 28, 2025 11:04

huynguyen-hl self-assigned this Feb 28, 2025

huynguyen-hl marked this pull request as draft February 28, 2025 12:45

feat: integrate MinIO for object store management in E2E tests

ee2c74e

huynguyen-hl force-pushed the feat/e2e-tests-workflow branch from b50eafd to ee2c74e Compare March 2, 2025 18:47

huynguyen-hl added 2 commits March 3, 2025 10:24

Merge remote-tracking branch 'origin/next' into feat/e2e-tests-workflow

e5829f8

feat: re-enable confetti test for valid credential in VCDM schema val…

369850e

…idation

huynguyen-hl marked this pull request as ready for review March 3, 2025 03:52

huynguyen-hl added 2 commits March 3, 2025 12:08

refactor: simplify clearObjectStore task parameters in common commands

1069355

Merge remote-tracking branch 'origin/next' into feat/e2e-tests-workflow

5dd138b

ashleythedeveloper reviewed Mar 3, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: implement e2e tests workflow #227

feat: implement e2e tests workflow #227

huynguyen-hl commented Feb 28, 2025 •

edited

Loading

github-actions bot commented Feb 28, 2025 •

edited

Loading

absoludity commented Mar 3, 2025 •

edited

Loading

huynguyen-hl commented Mar 3, 2025 •

edited

Loading

huynguyen-hl commented Mar 3, 2025 •

edited

Loading

ashleythedeveloper commented Mar 3, 2025 •

edited

Loading

ashleythedeveloper Mar 3, 2025

ashleythedeveloper Mar 3, 2025

ashleythedeveloper Mar 3, 2025

ashleythedeveloper Mar 3, 2025

ashleythedeveloper Mar 3, 2025

feat: implement e2e tests workflow #227

Are you sure you want to change the base?

feat: implement e2e tests workflow #227

Conversation

huynguyen-hl commented Feb 28, 2025 • edited Loading

What type of PR is this? (check all applicable)

Description

Mobile & Desktop Screenshots/Recordings

Testing E2E pipeline in Github personal account:

Added tests?

Added to documentation?

github-actions bot commented Feb 28, 2025 • edited Loading

Code Coverage Report

absoludity commented Mar 3, 2025 • edited Loading

huynguyen-hl commented Mar 3, 2025 • edited Loading

huynguyen-hl commented Mar 3, 2025 • edited Loading

ashleythedeveloper commented Mar 3, 2025 • edited Loading

ashleythedeveloper Mar 3, 2025

Choose a reason for hiding this comment

ashleythedeveloper Mar 3, 2025

Choose a reason for hiding this comment

ashleythedeveloper Mar 3, 2025

Choose a reason for hiding this comment

ashleythedeveloper Mar 3, 2025

Choose a reason for hiding this comment

ashleythedeveloper Mar 3, 2025

Choose a reason for hiding this comment

huynguyen-hl commented Feb 28, 2025 •

edited

Loading

github-actions bot commented Feb 28, 2025 •

edited

Loading

absoludity commented Mar 3, 2025 •

edited

Loading

huynguyen-hl commented Mar 3, 2025 •

edited

Loading

huynguyen-hl commented Mar 3, 2025 •

edited

Loading

ashleythedeveloper commented Mar 3, 2025 •

edited

Loading