Github Actions with Playwright + Synpress as plugin?

[DavitMkhitaryan]

Currently the tests on CI/CD fail on metamask.acceptAccess() step after clicking a button on the website which opens a metamask window to accept access. The error is on switch to metamask windows step. The tests run fine locally both headed and headless mode.

Error: [switchToMetamaskNotification] Max amount of retries to switch to metamask notification window has been reached. It was never found.

at Object.switchToMetamaskNotification (/home/runner/work/test-automation/test-automation/node_modules/@synthetixio/synpress/commands/playwright.js:193:13)

[DavitMkhitaryan]

Seems like the metamask setup is done correctly, but it is unable to switch to metamask notification window.

[duckception]

I cannot make this run on CI at all. Stuck on the Test timeout of 30000ms exceeded while setting up "context". error. How did you get through it? Also, what's the ENVIRONMENT_URL env variable in main.yml for?

[DavitMkhitaryan]

@duckception ENVIRONMENT_URL is not used, I just quickly put together a dummy repository on an example project. I just ran the "tests" workflow manually through Github Actions. I reproduced the issue again here: https://github.com/DavitMkhitaryan/test-automation-playwright-synpress/actions/runs/6105851115/job/16569960827

You can try again, I increased the timeout for tests.

[DavitMkhitaryan]

@duckception I have also invited you to the repo, so you can run the workflow directly from there.

[duckception]

I believe it doesn't work on CI because it lacks a screen to render the MetaMask window. You'll have to do something similar to this: https://github.com/Synthetixio/synpress/blob/dev/.github/workflows/e2e_headful.yml

[DavitMkhitaryan]

@duckception Are you saying that I can only run in the headful mode on the CI?

[DavitMkhitaryan]

Ok, it works in headful mode with a little adjustment to the github workflow file:

`name: tests

on:
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}

jobs:
dummy-test:
timeout-minutes: 60
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version: 18
- name: Install dependencies
run: npm install
- name: Install Playwright Browsers
run: npx playwright install --with-deps chromium
- name: Run Playwright tests
run: xvfb-run --auto-servernum -- npx playwright test
env:
ENVIRONMENT_URL: ${{ secrets.ENVIRONMENT_URL }}
HEADLESS_MODE: ${{secrets.HEADLESS_MODE}}
METAMASK_SETUP_NETWORK: ${{secrets.METAMASK_SETUP_NETWORK}}
METAMASK_SETUP_PASSWORD: ${{secrets.METAMASK_SETUP_PASSWORD}}
METAMASK_SETUP_PRIVATE_KEY: ${{secrets.METAMASK_SETUP_PRIVATE_KEY}}
- uses: actions/upload-artifact@v3
if: always()
with:
name: playwright-report
path: playwright-report/
retention-days: 30`

namely the 'xvfb-run --auto-servernum -- npx playwright test'

[DavitMkhitaryan]

Thanks @duckception ! Please also let me know if it's still not possible to run the tests in headless mode on the CI in Github Actions.

[duckception]

Currently, only headful mode is supported on CI 🫡