feat(testing): add Playwright accessibility testing POC for Badge and Status Light

[cdransf]

Description

SWC-1263

Proof of concept implementing automated accessibility testing using Playwright with two complementary approaches:

1. ARIA Snapshot Testing

• Captures and validates accessibility tree structure
• Detects unintended changes to component semantics
• Human-readable YAML format serves as living documentation

2. aXe-core WCAG Validation

• Automated WCAG 2.0/2.1 Level A/AA compliance checking
• Validates color contrast, ARIA attributes, keyboard accessibility
• Focused on standards compliance (excludes best-practice rules)

Test Coverage

25 tests passing across both generations (~30s runtime):

1st Generation (13 tests):

• Badge: 6 tests (3 ARIA snapshots, 3 aXe validations)
• Status Light: 7 tests (3 ARIA snapshots, 4 aXe validations)

2nd Generation (12 tests):

• Badge: 4 tests (2 ARIA snapshots, 2 aXe validations)
• Status Light: 8 tests (3 ARIA snapshots, 5 aXe validations)

Key Features

• Dual Storybook Support: Playwright projects handle both generations (1st gen: port 8080, 2nd gen: port 6006)
• Deterministic Testing: No arbitrary timeouts (waits for custom element definition, visibility, upgrade)
• Collocated Tests: Tests live with components in test/*.a11y.spec.ts
• Shared Helpers: Generation-agnostic utilities in first-gen/test/a11y-helpers.ts work for both gens
• Auto-start Storybooks: Both instances launch automatically when running tests
• Consolidated Documentation: Single source of truth at ACCESSIBILITY_TESTING.md

Usage

yarn test:a11y          # Run all tests (both generations)
yarn test:a11y:1st      # Only 1st gen tests
yarn test:a11y:2nd      # Only 2nd gen tests
yarn test:a11y:ui       # Interactive Playwright UI

Files Added/Modified

Test Files:

• first-gen/packages/badge/test/badge.a11y.spec.ts
• first-gen/packages/status-light/test/status-light.a11y.spec.ts
• second-gen/packages/swc/components/badge/test/badge.a11y.spec.ts
• second-gen/packages/swc/components/status-light/test/status-light.a11y.spec.ts

Shared Utilities:

• first-gen/test/a11y-helpers.ts - Deterministic helpers for both generations

Configuration:

• first-gen/playwright.config.ts - Dual-Storybook Playwright setup with projects
• first-gen/package.json - Test scripts (test:a11y, test:a11y:1st, test:a11y:2nd, test:a11y:ui)
• package.json - Root-level convenience commands

Documentation:

• ACCESSIBILITY_TESTING.md - Comprehensive guide covering quick start, how-to, and best practices

ARIA Snapshots (auto-generated baselines):

• first-gen/packages/badge/test/badge.a11y.spec.ts-snapshots/*.aria.yml
• first-gen/packages/status-light/test/status-light.a11y.spec.ts-snapshots/*.aria.yml
• second-gen/packages/swc/components/badge/test/badge.a11y.spec.ts-snapshots/*.aria.yml
• second-gen/packages/swc/components/status-light/test/status-light.a11y.spec.ts-snapshots/*.aria.yml

Motivation and Context

Problem: Manual accessibility testing is time-consuming and inconsistent. Regressions can slip through unnoticed.

Solution: Automated testing catches accessibility issues early in development and serves as executable documentation of expected behavior.

Approach: This POC demonstrates the pattern on Badge and Status Light components across both 1st and 2nd generation. If approved, the pattern can be applied to all components.

Technical Decisions

1. WCAG-only scanning

• Excludes best-practice rules (e.g., "page must have h1")
• Focuses on WCAG 2.0/2.1 Level A/AA compliance
• Reasoning: We test isolated components in Storybook, not full pages

2. Deterministic waits

• Custom gotoStory() helper waits for specific conditions
• No arbitrary timeouts or waitForLoadState('networkidle')
• Faster, more reliable than waiting for network idle

3. Playwright projects for dual-generation support

• Separate configurations for 1st gen (port 8080) and 2nd gen (port 6006)
• Allows running tests independently or together
• Auto-starts both Storybook instances as needed

4. Collocated tests

• Tests live in each component's test/ directory
• Follows existing test organization patterns
• Easier discovery and maintenance

5. Shared, generation-agnostic helpers

• Single helper library at first-gen/test/a11y-helpers.ts
• Works for both 1st gen (sp-*) and 2nd gen (swc-*) components
• No code duplication

Related Issue(s)

• This is a proof-of-concept RFC for team discussion
• No existing issue (greenfield implementation)

---

Author's Checklist

• I have read the CONTRIBUTING and PULL_REQUESTS documents
• I have reviewed the Accessibility Practices for this feature
• I have added automated tests to cover my changes
• I have included updated documentation

---

Reviewer's Checklist

Questions for Discussion

1. Approach: Do ARIA snapshots + aXe validation provide the right balance?
2. Coverage: Should we add these tests to all components or start with high-priority ones?
3. CI Integration: How should these run in CI? (All PRs? Scheduled? On-demand?)
4. Maintenance: Who owns keeping snapshots updated when designs change?
5. Snapshot Management: ARIA snapshots are committed - is this the right approach?

Manual Testing

Verify tests run successfully:

1. Clean environment: pkill -f "storybook"
2. Run all tests: cd first-gen && yarn test:a11y
3. Expected: 25 passing tests in ~30 seconds
4. Verify both Storybooks auto-start (ports 8080 and 6006)

Review test outputs:

1. Open HTML report: yarn playwright show-report
2. Review ARIA snapshots in **/test/*-snapshots/ directories
3. Verify human-readable YAML format

Try individual commands:

• yarn test:a11y:1st - Should run 13 1st gen tests
• yarn test:a11y:2nd - Should run 12 2nd gen tests
• yarn test:a11y:ui - Should open Playwright UI
• yarn test:a11y badge --update-snapshots - Update baselines

Review documentation:

• Read ACCESSIBILITY_TESTING.md for comprehensive guide
• Follow "Adding tests to a component" section
• Verify examples are clear and copy-paste ready

---

Note: This is a POC for discussion. Not intended for immediate merge - seeking feedback on approach and implementation before broader rollout.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 4f3b5ed

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[github-actions]

📚 Branch Preview

• Documentation Site
• Storybook

🔍 Visual Regression Test Results

When a visual regression test fails (or has previously failed while working on this branch), its results can be found in the following URLs:

• Spectrum | Light | Medium | LTR
• Spectrum | Dark | Large | RTL
• Express | Light | Medium | LTR
• Express | Dark | Large | RTL
• Spectrum-two | Light | Medium | LTR
• Spectrum-two | Dark | Large | RTL
• High Contrast Mode | Medium | LTR

Deployed to Azure Blob Storage: pr-5835

If the changes are expected, update the current_golden_images_cache hash in the circleci config to accept the new images. Instructions are included in that file.
If the changes are unexpected, you can investigate the cause of the differences and update the code accordingly.

[github-actions]

Tachometer results

Currently, no packages are changed by this PR...