Record Playwright tests with Replay #348

bvaughn · 2024-05-01T14:55:23Z

No description provided.

vercel · 2024-05-01T14:55:29Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
react-resizable-panels	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 19, 2024 1:50pm

Andarist · 2024-05-02T12:33:38Z

package.json

@@ -23,6 +23,7 @@
  "devDependencies": {
    "@babel/preset-typescript": "^7.22.5",
    "@playwright/test": "^1.37.0",
+    "@replayio/playwright": "3.0.0-alpha.3",


does this setup work for you with this version now?

No. The latest comment I left in Linear mentioned that the steps are all missing. I'll add you to the workspace so you can look at them.

Let's see what happens with the newest version ( #349 ). I'm simultaneously testing this release in the overboard's repo

Here's the latest recording: https://app.replay.io/recording/68393c17-bc53-4c80-a972-f29c9e0fa579

No steps.

I'm able to repro this locally using ur tests here... diving in! 🤿

packages/react-resizable-panels-website/playwright.config.ts

bvaughn · 2024-05-03T16:05:02Z

@Andarist The latest v6 alpha looks to have both steps and metadata. Great!

bvaughn · 2024-05-03T16:10:52Z

Removing the test.only so that all of my tests run shows a new failure though:

RecordingUploadError: Failed to set recording metadata Error: Protocol client is not initialized

Andarist · 2024-05-03T16:36:59Z

Interesting, gonna investigate soon!

Andarist · 2024-05-06T12:29:17Z

package.json

@@ -23,6 +23,7 @@
  "devDependencies": {
    "@babel/preset-typescript": "^7.22.5",
    "@playwright/test": "^1.37.0",
+    "@replayio/playwright": "3.0.0-alpha.6",


this could now be bumped to .7

Gotcha. Just bumped the version.

Noticed that I can't run e2e tests locally though:

$ p test:e2e > [email protected] test:e2e /Users/bvaughn/Documents/git/oss/react-resizable-panels/packages/react-resizable-panels-website > playwright test [replay.io]: Failed to send test metrics [replay.io]: Error: request to https://webhooks.replay.io/api/metrics failed, reason: socket hang up [replay.io]: 🕑 Completing some outstanding work ... [replay.io]: [replay.io]: ❌ We encountered some unexpected errors processing your recordings and was unable to upload them. [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io [replay.io]: - Failed to upload recording: Cannot connect to server wss://dispatch.replay.io

Maybe this is because I didn't pass the api token env var? But the error message isn't great if that's the case, and this would mean that no contributors could run these tests either.

Also the terminal was totally silent for several seconds before printing this. I'm not sure if that meant tests weren't running, or if it was just swallowing the stdout output. (That output is important to debug.)

So you have this error, CI reports "RecordingUploadError: Failed to set recording metadata Error: Protocol client is not initialized" and I have "RecordingUploadError: Failed to set recording metadata Protocol error 1: Internal error" locally now 😂

Well, I'm investigating all of this now ;p

This one makes sure that the `@replayio/reply` (transitive dependency) gets updated too (u can see the change in the lockfile). With this one, I can record this repo and I get recordings. Some fail at times when the metadata's size gets exceeded. We know about the issue and we are in the process of figuring out the solution.

Andarist · 2024-05-09T13:19:42Z

We have a partial success here! It still suffers from the metadata size problem - but most tests don't exceed that size and recordings are nicely created. Could you also confirm that it works for you locally?

bvaughn · 2024-05-09T14:28:12Z

Could you also confirm that it works for you locally?

Yes, tests do run (although tests/StackingOrder.spec seems perhaps a bit flakier? could be a coincidence or could be a difference caused by the older Chromium version).

I think our runner's output is less helpful as you go. For instance, running these tests on main gives feedback as you go:

Running them on this branch is more quiet:

But that's a nit.

This addresses the complaint from #348 (comment) . I didn't realize that the default is `'list'` locally, I added `'line'` reporter since I've seen that being used in a couple of other projects. I totally agree that `'list'` is quite nice and I prefer it to `'line'` - especially if we consider smaller test suites. ⚠️ `@replayio/playwright` still spits out a bunch of junk into the stdout. That makes the current output very cluttered - even with this improvement. This is going to be addressed with PRO-315

replay-io · 2024-05-19T13:50:52Z

Status	Complete ↗︎
Commit	`eaa041f`
Results	❌ 1 Failed should update cursor when dragged past the min/max size (like VS Code) ✅ 23 Passed earlier panels should be springy when expanding then collapsing the last panel Enter key (collapsible) Enter key (not collapsible) Home and End keys horizontal lists implements optional F6 key functionality later panels should be springy when expanding then collapsing the first panel panels should remember a max spring point per drag scenario: other maxSize(s) scenario: other minSize(s) scenario: own maxSize scenario: own minSize should call onResize when panels are resized should not update cursor or start dragging if a resize handle is underneath another element should not update cursor style for resizes caused by keyboard events should remember the most recent expanded size for collapsed panels should resize and maintain layouts independently should restore previous layout if autoSaveId prop has been provided should set 'data-resize-handle-active' attribute when active should store layouts separately per panel combination should support custom collapsedSize values should update cursor when dragging intersecting panels (like tmux) vertical lists

Record Playwright tests with Replay

fbfeae2

vercel bot deployed to Preview May 1, 2024 14:56 View deployment

Pass through secrets.REPLAY_API_KEY

f7b0c54

vercel bot deployed to Preview May 1, 2024 14:58 View deployment

Downgrade to alpha 3

307293d

vercel bot deployed to Preview May 1, 2024 16:09 View deployment

Set REPLAY_PLAYWRIGHT_FIXTURE

e4496d9

vercel bot deployed to Preview May 1, 2024 18:24 View deployment

Andarist reviewed May 2, 2024

View reviewed changes

Update @replayio/playwright (#349)

56f99f0

vercel bot deployed to Preview May 2, 2024 13:56 View deployment

bvaughn added 2 commits May 2, 2024 09:57

Lint

e9a97a5

TypeScript

d698a54

vercel bot deployed to Preview May 2, 2024 14:01 View deployment

Andarist reviewed May 2, 2024

View reviewed changes

packages/react-resizable-panels-website/playwright.config.ts Outdated Show resolved Hide resolved

Remove @ts-ignore

40faa4f

vercel bot deployed to Preview May 2, 2024 14:19 View deployment

Update to 3.0.0-alpha.6

976c710

vercel bot deployed to Preview May 3, 2024 16:03 View deployment

Remove test.only

9b50f64

vercel bot deployed to Preview May 3, 2024 16:06 View deployment

Andarist reviewed May 6, 2024

View reviewed changes

Update @replayio/playwright to v3 alpha 7

5f9ef18

vercel bot deployed to Preview May 6, 2024 12:47 View deployment

vercel bot deployed to Preview May 9, 2024 12:56 View deployment

Andarist mentioned this pull request May 10, 2024

Use a nicer list reporter locally #356

Merged

vercel bot deployed to Preview May 10, 2024 12:35 View deployment

Empty commit

eaa041f

vercel bot deployed to Preview May 19, 2024 13:50 View deployment

bvaughn closed this Jul 12, 2024

bvaughn deleted the replayio branch July 12, 2024 18:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Record Playwright tests with Replay #348

Record Playwright tests with Replay #348

bvaughn commented May 1, 2024

vercel bot commented May 1, 2024 •

edited

Loading

Andarist May 2, 2024

bvaughn May 2, 2024

Andarist May 2, 2024

bvaughn May 2, 2024

Andarist May 2, 2024

bvaughn commented May 3, 2024

bvaughn commented May 3, 2024

Andarist commented May 3, 2024

Andarist May 6, 2024

bvaughn May 6, 2024

bvaughn May 6, 2024

Andarist May 6, 2024

Andarist commented May 9, 2024

bvaughn commented May 9, 2024

replay-io bot commented May 19, 2024 •

edited

Loading

Record Playwright tests with Replay #348

Record Playwright tests with Replay #348

Conversation

bvaughn commented May 1, 2024

vercel bot commented May 1, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bvaughn commented May 3, 2024

bvaughn commented May 3, 2024

Andarist commented May 3, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Andarist commented May 9, 2024

bvaughn commented May 9, 2024

replay-io bot commented May 19, 2024 • edited Loading

vercel bot commented May 1, 2024 •

edited

Loading

replay-io bot commented May 19, 2024 •

edited

Loading