Playwright Snapshots blog #295

mnyamunda-scottlogic · 2025-01-24T16:24:18Z

Please add a direct link to your post here: https://mnyamunda-scottlogic.github.io/blog/2025/01/22/playwright-visual-testing.html

Have you (please tick each box to show completion):

Added your blog post to a single category?
Added a brief summary for your post? Summaries should be roughly two sentences in length and give potential readers a good idea of the contents of your post.
Checked that the build passes?
Checked your spelling (you can use npm install followed by npx mdspell "**/{FILE_NAME}.md" --en-gb -a -n -x -t if that's your thing)
Ensured that your author profile contains a profile image, and a brief description of yourself? (make it more interesting than just your job title!)
Optimised any images in your post? They should be less than 100KBytes as a general guide.

Posts are reviewed / approved by your Regional Tech Lead.

chriswilty · 2025-02-07T08:26:54Z

_posts/2025-01-09-playwright-visual-testing.markdown

+
+## The Problem
+
+The main challenge was to create a dashboard that shows discrepancies between forecasted air quality and the real time measurements. The main page was simple enough as it was using ag grid, hence we could mock and assert values within each cell. However, it would not be as simple with chart-like elements. We needed to test that the data in these charts was visualised correctly. With default mocked data and then secondly, we needed to test that the various UI interactions transform the charts correctly.


visualised correctly. With default mocked data

Don't think this full-stop should be there, and there's also a Secondly without a Firstly.

chriswilty · 2025-02-07T08:29:20Z

_posts/2025-01-09-playwright-visual-testing.markdown

+
+![Forecasted and in-situ air quality graphs for the city Seoul]({{ site.baseurl }}/mnyamunda/assets/aqi-charts.png)
+
+Shown above is the UI consisting of six different charts that show forecasted and in-situ pollutant measurements. The first big problem is that the data is non-static. We solve this by mocking data via Playwright. This way, our regression focuses on the functionality and various transformations that the data undergoes. The second problem is how do we know that the actual data is being visualised correctly? This is where Playwright snapshots come in.


"solve this by mocking the data API" maybe? Also a link to the docs would be helpful...
https://playwright.dev/docs/mock

chriswilty

A few comments. Many of the code blocks are messed up, don't know why...

chriswilty · 2025-02-07T08:40:26Z

_posts/2025-01-09-playwright-visual-testing.markdown

+## Playwright Snapshotting Explained
+
+Playwright will startup each browser and complete various actions and take a screenshot of the DOM element(s) state. It will then sequentially look at each pixel and check if it’s RGBA values match up to the comparison image created in the codebase.
+With Playwright snapshots we can easily assert how a page element should look. Provided that we don’t have a comparison image in the codebase our first run of `npx playwright test` will create a comparison image for us. Now we must tweak our test steps until we are satisfied with the snapshot created by playwright. Once satisfied we now run the same command with the flag `--update-snapshots` which then saves the snapshot as our golden standard referral image. Now each time we run the test we will compare against that image.


Playwright docs make a clear distinction between screenshots, and (aria) snapshots. In fact, the docs state this:

Are you using snapshots because you want to take a partial screenshot, of just the chart element?

That's correct, we specifically used it to capture the chart elements.

chriswilty · 2025-02-07T08:42:22Z

_posts/2025-01-09-playwright-visual-testing.markdown

+test('Verify so2 graph updates correctly when sites are removed', async ({
+    cityPage,
+  }) => {
+    await cityPage.siteRemover('Centro')


What's that for then? If it's not relevant to the post, remove it, else explain what you're doing?

Okay, I see what this is for now, having read the rest of the post, but up here I did not catch on from the name of the test. Might be better to simplify this test example just to capture the snapshot and verify it.

That's a fair point, i will adda comment to indicate what the step is actually doing 💯

chriswilty · 2025-02-07T08:46:13Z

_posts/2025-01-09-playwright-visual-testing.markdown

+
+Shown above is the UI consisting of six different charts that show forecasted and in-situ pollutant measurements. The first big problem is that the data is non-static. We solve this by mocking data via Playwright. This way, our regression focuses on the functionality and various transformations that the data undergoes. The second problem is how do we know that the actual data is being visualised correctly? This is where Playwright snapshots come in.
+
+## Playwright Snapshotting Explained


I can't find any reference to the term Snapshotting, maybe best leave it as Snapshot Testing.

Sounds good to me

chriswilty · 2025-02-07T08:48:49Z

_posts/2025-01-09-playwright-visual-testing.markdown

+
+## Playwright Snapshotting Explained
+
+Playwright will startup each browser and complete various actions and take a screenshot of the DOM element(s) state. It will then sequentially look at each pixel and check if it’s RGBA values match up to the comparison image created in the codebase.


complete various actions

A bit woolly. What actions? Authentication, navigate to page and wait for it to load?

True, will give some examples!

chriswilty · 2025-02-07T08:52:14Z

_posts/2025-01-09-playwright-visual-testing.markdown

+
+Without any additional parameters, the comparison will fail if a single pixel is different. This option allows for maximum accuracy.
+
+```


I have no idea why this is not rendering correctly 🤔

chriswilty · 2025-02-07T08:53:40Z

_posts/2025-01-09-playwright-visual-testing.markdown

+
+“maxDiffPixels”
+
+`in-situ-AQI-is-4-at-00:00-due-to-PM2.5.png,{ maxDiffPixels:27 }`


I suggest including function name, toMatchSnapshot(

Otherwise it's not clear where these options are supposed to go.

I agree with that, will update!

chriswilty · 2025-02-07T08:57:15Z

_posts/2025-01-09-playwright-visual-testing.markdown

+
+This is instead a percentage check. If up to 27% of the pixels do not match, then the test will still pass. Upon running the test through playwright CLI commands, a report is generated at the end of the test. This will show us what is different in the image.
+
+## What does failure look like?


Your previous headings are ## All Words Propercased as opposed to ## First word capitalized like a sentence. Might be better to stick to one or other, for consistency.

True! Will rectify this.

chriswilty · 2025-02-07T08:59:02Z

_posts/2025-01-09-playwright-visual-testing.markdown

+
+Expected:
+
+![sulphur dioxide polution level chart expectation]({{ site.baseurl }}/mnyamunda/assets/sulphur-dioxide-chart-expected.png)


Typo: pollution

I assume these are the broken/missing images you mentioned in teams?

Good spot, broken images are now fixed. Will correct this!

chriswilty · 2025-02-07T09:05:12Z

_posts/2025-01-09-playwright-visual-testing.markdown

+- Saves on some lengthy manual work. As shown above we could have to manually check and click around on 6 different charts which would be very time consuming if regression is carried out on a per-ticket basis.
+- Sometimes you can expect the element to be visible but the actual contents of it may be misrepresented or broken.
+
+  ![Broken image icon]({{ site.baseurl }}/mnyamunda/assets/broken-image-icon.png)


Unsure what this image is portraying, as there is nothing for comparison.

Hey Chris, this might not actually add much value. It was just supposed to be a broken image png

chriswilty · 2025-02-07T09:07:55Z

_posts/2025-01-09-playwright-visual-testing.markdown

+    const siteDeselect = this.page.getByLabel(`Remove ${location}`)
+    await siteDeselect.click()
+  }
+```


Again, these code blocks are broken in the current view:

mnyamunda-scottlogic added 21 commits January 9, 2025 08:27

Initial blog setup

71006f8

Playwright snapshots blog

b3ba656

remove accidental changes

b7a9f26

Restore accidental deletion

6f05f75

single quote to double quote

8eb3a35

Minor restoration (final)

01a4af6

update author-summary

c7be43e

Auto format fix

fe458e5

typo fix

f447f94

Add tags, create image assets folder

0652180

update image locations

6745d0f

Spelling fixes

376322c

compress images

267aac1

remove extra line

771fbc5

rename image, update imageURL

0b6219f

change image names, update file path method

cbb3a61

Update Role

1f0d587

fix typo in title

20fbf52

change asset paths to not be github username

17a9ea6

number to word

c0bb71b

Update author image path, update code blocks

b0dd69e

chriswilty reviewed Feb 7, 2025

View reviewed changes

Update png name

f746600

chriswilty reviewed Feb 7, 2025

View reviewed changes

mnyamunda-scottlogic added 5 commits February 7, 2025 10:47

code block update

49e404b

correct image names, format code blocks

d9ea144

code block highlighting to ts

e0b4d96

Remove broken image, add syntaz highlighting

4605c51

correct typo, consistent headings, update code snip for more clarity

0252ee0

mnyamunda-scottlogic added 5 commits February 7, 2025 14:32

clarify "various actions"

4c515d6

update test with comments, add link to playwright mock apis

b4fe52e

add missing dash to flag

6d3e943

Title change

3a9370d

Grammatical update

e313c7d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Playwright Snapshots blog #295

Playwright Snapshots blog #295

mnyamunda-scottlogic commented Jan 24, 2025 •

edited

Loading

chriswilty Feb 7, 2025 •

edited

Loading

mnyamunda-scottlogic Feb 10, 2025

chriswilty Feb 7, 2025

mnyamunda-scottlogic Feb 10, 2025

chriswilty left a comment

chriswilty Feb 7, 2025

mnyamunda-scottlogic Feb 7, 2025

chriswilty Feb 7, 2025

chriswilty Feb 7, 2025

mnyamunda-scottlogic Feb 7, 2025

chriswilty Feb 7, 2025

mnyamunda-scottlogic Feb 7, 2025

chriswilty Feb 7, 2025

mnyamunda-scottlogic Feb 7, 2025

chriswilty Feb 7, 2025

chriswilty Feb 7, 2025

mnyamunda-scottlogic Feb 7, 2025

chriswilty Feb 7, 2025

mnyamunda-scottlogic Feb 7, 2025

chriswilty Feb 7, 2025

mnyamunda-scottlogic Feb 7, 2025

chriswilty Feb 7, 2025

mnyamunda-scottlogic Feb 7, 2025

chriswilty Feb 7, 2025 •

edited

Loading


		## The Problem

		The main challenge was to create a dashboard that shows discrepancies between forecasted air quality and the real time measurements. The main page was simple enough as it was using ag grid, hence we could mock and assert values within each cell. However, it would not be as simple with chart-like elements. We needed to test that the data in these charts was visualised correctly. With default mocked data and then secondly, we needed to test that the various UI interactions transform the charts correctly.


		![Forecasted and in-situ air quality graphs for the city Seoul]({{ site.baseurl }}/mnyamunda/assets/aqi-charts.png)

		Shown above is the UI consisting of six different charts that show forecasted and in-situ pollutant measurements. The first big problem is that the data is non-static. We solve this by mocking data via Playwright. This way, our regression focuses on the functionality and various transformations that the data undergoes. The second problem is how do we know that the actual data is being visualised correctly? This is where Playwright snapshots come in.


		Shown above is the UI consisting of six different charts that show forecasted and in-situ pollutant measurements. The first big problem is that the data is non-static. We solve this by mocking data via Playwright. This way, our regression focuses on the functionality and various transformations that the data undergoes. The second problem is how do we know that the actual data is being visualised correctly? This is where Playwright snapshots come in.

		## Playwright Snapshotting Explained


		## Playwright Snapshotting Explained

		Playwright will startup each browser and complete various actions and take a screenshot of the DOM element(s) state. It will then sequentially look at each pixel and check if it’s RGBA values match up to the comparison image created in the codebase.


		Without any additional parameters, the comparison will fail if a single pixel is different. This option allows for maximum accuracy.

		```


		“maxDiffPixels”

		`in-situ-AQI-is-4-at-00:00-due-to-PM2.5.png,{ maxDiffPixels:27 }`


		This is instead a percentage check. If up to 27% of the pixels do not match, then the test will still pass. Upon running the test through playwright CLI commands, a report is generated at the end of the test. This will show us what is different in the image.

		## What does failure look like?


		Expected:

		![sulphur dioxide polution level chart expectation]({{ site.baseurl }}/mnyamunda/assets/sulphur-dioxide-chart-expected.png)

Playwright Snapshots blog #295

Are you sure you want to change the base?

Playwright Snapshots blog #295

Conversation

mnyamunda-scottlogic commented Jan 24, 2025 • edited Loading

chriswilty Feb 7, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chriswilty left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chriswilty Feb 7, 2025 • edited Loading

Choose a reason for hiding this comment

mnyamunda-scottlogic commented Jan 24, 2025 •

edited

Loading

chriswilty Feb 7, 2025 •

edited

Loading

chriswilty Feb 7, 2025 •

edited

Loading