[Axe] Run axe across all stories #5592

kendallgassner · 2025-01-23T21:32:26Z

In follow up PR:

Remove old @aat tests

This PR aims to run axe ci testing on all Storybook stories. I noticed in a previous PR that we had uncaught axe violations because we have to manually add a Playwright test for each Storybook story. Because of this manual effort many of our stories do not have axe testing setup.

In addition to writing a script to run axe against all stories I made fixes to all pre-existing stories with axe failures.

Changelog

New

Changed

Removed

Rollout strategy

Patch release
Minor release
Major release; if selected, include a written rollout or migration plan
None; if selected, include a brief description as to why

Testing & Reviewing

Merge checklist

Added/updated tests
Added/updated documentation
Added/updated previews (Storybook)
Changes are SSR compatible
Tested in Chrome
Tested in Firefox
Tested in Safari
Tested in Edge
(GitHub staff only) Integration tests pass at github/github (Learn more about how to run integration tests)

changeset-bot · 2025-01-23T21:32:30Z

⚠️ No Changeset found

Latest commit: e492c51

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

github-actions · 2025-01-23T21:51:30Z

👋 Hi, this pull request contains changes to the source code that github/github depends on. If you are GitHub staff, we recommend testing these changes with github/github using the integration workflow. Thanks!

github-actions · 2025-01-23T21:54:18Z

size-limit report 📦

Path	Size
packages/react/dist/browser.esm.js	104.56 KB (0%)
packages/react/dist/browser.umd.js	104.89 KB (0%)

kendallgassner · 2025-01-28T19:03:40Z

packages/react/src/Banner/Banner.stories.tsx

@@ -49,6 +50,7 @@ export const Playground: StoryObj<typeof Banner> = {

        <PageLayout.Content>
          <Banner
+            aria-label="Content level banner"


Unique aria-labels!

kendallgassner · 2025-01-28T19:04:40Z

In follow up PRs:

work to remove individual aat tests.
consider automating VRT tests.

joshblack

Looks great! Just left a couple of comments 👀

joshblack · 2025-01-28T20:24:24Z

e2e/components/Axe.test.ts

+
+const {entries} = componentsConfig
+
+test.describe('@aat', () => {


Suggested change

test.describe('@aat', () => {

test.describe('aat', () => {

I would just annotate this as aat, or remove it, since we're using the @aat tag at the test-level to filter things out currently (although we could totally change this in the future.

My thought was to remove all @aat tests after this is merged since we don't need to generate them ourselves

@joshblack do you feel like we need the other @Aat tests now?

@kendallgassner I totally bet we could, especially the automated/boilerplate ones.

I think it can still be helpful to have this tag since there are stories where we would like to put the component in a specific state and then run axe, ideally. Like with menus to make sure we test things when it's open and closed or if we want to use the matrix test helper to make sure we run axe against prop combinations of things.

@joshblack If we keep the '@Aat' in this file, doesn't it break up the tests to run concurrently in aat-report. I assumed aat-runner 1-4 were different threads.

e2e/components/Axe.test.ts

Co-authored-by: Josh Black <[email protected]>

kendallgassner · 2025-01-28T22:07:33Z

A legit color-contrast failure: flash banner with a link in it. I might need to add the components-flash-features--with-icon-action-dismiss story to the skip list until we can get a fix out for this.

primer-integration · 2025-01-29T17:41:34Z

👋 Hi from github/github! Your integration PR is ready: https://github.com/github/github/pull/359958

TylerJDev · 2025-01-30T14:51:40Z

Random question; I see the flow isn't part of the workflow ci.yml anymore, and was wondering if this was intentional? Just trying to see where the tests are being ran in the CI 👀

kendallgassner · 2025-01-30T17:01:59Z

@TylerJDev I didn't have to change the CI because @joshblack pointed out that I could use the storybook-static/index.json file to pull storybook ids ✨ . This file is generated during npx storybook build which is already run during the aat report!

The aat report will run these specific tests because of the @aat tag.

kendallgassner added 7 commits January 21, 2025 20:05

Run storybook axe scans on CI

a134660

test

6d3928a

clean

5517189

remove console

87a95d2

rephrase id

dc061f8

unused

154e062

edits

5ebec1d

Merge branch 'main' into kendallg/axe-test-on-ci

3d1494d

github-actions bot added the integration-tests: recommended This change needs to be tested for breaking changes. See https://arc.net/l/quote/tdmpakpm label Jan 23, 2025

kendallgassner temporarily deployed to github-pages January 23, 2025 21:52 — with GitHub Actions Inactive

github-actions bot temporarily deployed to storybook-preview-5592 January 23, 2025 21:53 Inactive

kendallgassner added the skip changeset This change does not need a changelog label Jan 23, 2025

try

05bff8b

kendallgassner temporarily deployed to github-pages January 23, 2025 22:01 — with GitHub Actions Inactive

github-actions bot temporarily deployed to storybook-preview-5592 January 23, 2025 22:01 Inactive

kendallgassner added 2 commits January 23, 2025 14:55

test

9c29e5e

test

a07320c

kendallgassner temporarily deployed to github-pages January 23, 2025 22:59 — with GitHub Actions Inactive

github-actions bot temporarily deployed to storybook-preview-5592 January 23, 2025 22:59 Inactive

store

3b3a0b0

kendallgassner temporarily deployed to github-pages January 23, 2025 23:09 — with GitHub Actions Inactive

github-actions bot temporarily deployed to storybook-preview-5592 January 23, 2025 23:09 Inactive

store

35aefd2

kendallgassner temporarily deployed to github-pages January 23, 2025 23:15 — with GitHub Actions Inactive

github-actions bot temporarily deployed to storybook-preview-5592 January 23, 2025 23:16 Inactive

store

59dbd1b

kendallgassner temporarily deployed to github-pages January 23, 2025 23:21 — with GitHub Actions Inactive

kendallgassner commented Jan 28, 2025

View reviewed changes

kendallgassner temporarily deployed to github-pages January 28, 2025 19:10 — with GitHub Actions Inactive

github-actions bot temporarily deployed to storybook-preview-5592 January 28, 2025 19:10 Inactive

kendallgassner marked this pull request as ready for review January 28, 2025 20:15

kendallgassner requested a review from a team as a code owner January 28, 2025 20:15

kendallgassner requested review from camertron and joshblack January 28, 2025 20:15

joshblack reviewed Jan 28, 2025

View reviewed changes

Update e2e/components/Axe.test.ts

40929ea

Co-authored-by: Josh Black <[email protected]>

kendallgassner temporarily deployed to github-pages January 28, 2025 20:55 — with GitHub Actions Inactive

github-actions bot temporarily deployed to storybook-preview-5592 January 28, 2025 20:55 Inactive

add test skip

62b00f8

kendallgassner temporarily deployed to github-pages January 28, 2025 23:09 — with GitHub Actions Inactive

github-actions bot temporarily deployed to storybook-preview-5592 January 28, 2025 23:09 Inactive

add test skip

cb1c7b4

kendallgassner temporarily deployed to github-pages January 28, 2025 23:42 — with GitHub Actions Inactive

github-actions bot temporarily deployed to storybook-preview-5592 January 28, 2025 23:43 Inactive

Merge branch 'main' into kendallg/axe-test-on-ci

6912356

kendallgassner temporarily deployed to github-pages January 29, 2025 17:39 — with GitHub Actions Inactive

github-actions bot temporarily deployed to storybook-preview-5592 January 29, 2025 17:39 Inactive

add progressbar fix

e492c51

kendallgassner deployed to github-pages January 29, 2025 17:57 — with GitHub Actions View deployment

github-actions bot deployed to storybook-preview-5592 January 29, 2025 17:58 View deployment

kendallgassner requested review from khiga8 and TylerJDev January 29, 2025 18:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Axe] Run axe across all stories #5592

[Axe] Run axe across all stories #5592

kendallgassner commented Jan 23, 2025 •

edited

Loading

changeset-bot bot commented Jan 23, 2025 •

edited

Loading

github-actions bot commented Jan 23, 2025

github-actions bot commented Jan 23, 2025 •

edited

Loading

kendallgassner Jan 28, 2025

kendallgassner commented Jan 28, 2025

joshblack left a comment

joshblack Jan 28, 2025

kendallgassner Jan 28, 2025

kendallgassner Jan 28, 2025

joshblack Jan 28, 2025

kendallgassner Jan 28, 2025 •

edited

Loading

kendallgassner commented Jan 28, 2025 •

edited

Loading

primer-integration bot commented Jan 29, 2025

TylerJDev commented Jan 30, 2025

kendallgassner commented Jan 30, 2025


		const {entries} = componentsConfig

		test.describe('@aat', () => {

[Axe] Run axe across all stories #5592

Are you sure you want to change the base?

[Axe] Run axe across all stories #5592

Conversation

kendallgassner commented Jan 23, 2025 • edited Loading

Changelog

New

Changed

Removed

Rollout strategy

Testing & Reviewing

Merge checklist

changeset-bot bot commented Jan 23, 2025 • edited Loading

⚠️ No Changeset found

github-actions bot commented Jan 23, 2025

github-actions bot commented Jan 23, 2025 • edited Loading

size-limit report 📦

kendallgassner Jan 28, 2025

Choose a reason for hiding this comment

kendallgassner commented Jan 28, 2025

joshblack left a comment

Choose a reason for hiding this comment

joshblack Jan 28, 2025

Choose a reason for hiding this comment

kendallgassner Jan 28, 2025

Choose a reason for hiding this comment

kendallgassner Jan 28, 2025

Choose a reason for hiding this comment

joshblack Jan 28, 2025

Choose a reason for hiding this comment

kendallgassner Jan 28, 2025 • edited Loading

Choose a reason for hiding this comment

kendallgassner commented Jan 28, 2025 • edited Loading

primer-integration bot commented Jan 29, 2025

TylerJDev commented Jan 30, 2025

kendallgassner commented Jan 30, 2025

kendallgassner commented Jan 23, 2025 •

edited

Loading

changeset-bot bot commented Jan 23, 2025 •

edited

Loading

github-actions bot commented Jan 23, 2025 •

edited

Loading

kendallgassner Jan 28, 2025 •

edited

Loading

kendallgassner commented Jan 28, 2025 •

edited

Loading