# Automating axe-core Form Audits in CI

This recipe wires axe-core form-accessibility audits into a GitHub Actions pipeline so that any new WCAG violation in a form’s error state fails the build, with baseline management to avoid blocking on pre-existing debt.

# When to Use This Recipe

Reach for CI-enforced audits once your form validation has accessible-error wiring worth protecting — aria-invalid, aria-describedby, and live regions, as produced by the native Constraint Validation API Deep Dive. It is the right tool when you want regressions caught at the pull request, not in QA. If you only need to write the scans themselves, start with axe-core Accessibility Testing first; this recipe assumes those scans exist and focuses on automation, build-gating, and baselines.

# Minimal Working Implementation

The scan itself lives in a Playwright test that drives the form into its error state and audits only the form subtree. Failing on new violations — rather than all violations — is what makes the gate adoptable on an existing codebase.

// tests/a11y/signup.a11y.spec.ts
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';
import { readFileSync } from 'node:fs';

// Rule ids known to fail today, accepted as debt until fixed.
const baseline: string[] = JSON.parse(
  readFileSync(new URL('./axe-baseline.json', import.meta.url), 'utf8'),
);

test('signup form error state introduces no new a11y violations', async ({ page }) => {
  await page.goto('/signup');

  // Trigger validation so axe audits the real error DOM.
  await page.getByRole('button', { name: 'Create Account' }).click();
  await expect(page.getByText(/required/i).first()).toBeVisible();

  const { violations } = await new AxeBuilder({ page })
    .include('#signup-form')                       // scope to the form
    .withTags(['wcag2a', 'wcag2aa', 'wcag22aa'])
    .analyze();

  // Only fail on violations not already accepted in the baseline.
  const regressions = violations.filter((v) => !baseline.includes(v.id));

  if (regressions.length) {
    console.error(
      regressions.map((v) => `${v.id} (${v.impact}): ${v.help}`).join('\n'),
    );
  }
  expect(regressions, 'new accessibility violations').toEqual([]);
});

The companion axe-baseline.json is just an array of rule ids you have consciously deferred:

// tests/a11y/axe-baseline.json
["color-contrast"]

The GitHub Actions workflow runs the suite and fails the job on any non-zero Playwright exit. Caching the browser binaries keeps the accessibility job under a minute.

# .github/workflows/a11y.yml
name: a11y-audit
on: [pull_request]
jobs:
  axe-form-audit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - name: Cache Playwright browsers
        uses: actions/cache@v4
        with:
          path: ~/.cache/ms-playwright
          key: pw-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
      - run: npx playwright install --with-deps chromium
      # Non-zero exit on any regression fails the merge.
      - run: npx playwright test tests/a11y --reporter=line
      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: playwright-report
          path: playwright-report/

# Parameter Reference

# Verification Steps

Confirm the gate actually blocks regressions before trusting it. Introduce a deliberate violation — remove a <label> or lower error contrast — push the branch, and watch the axe-form-audit job turn red with the offending rule id in the log. Then revert and confirm green. Locally, reproduce CI with:

npx playwright test tests/a11y --reporter=line

Inspect the uploaded playwright-report artifact on a failed run to see each violation’s selector and failureSummary.

# Edge Cases & Failure Modes

The scan runs before an async error renders. A debounced server check populates the live region after the scan, so the audit passes against an empty container. Always await expect(...).toBeVisible() on the error before calling .analyze(), as shown above. This is the same race that motivates cancelling stale requests in async validation flows.

The baseline silently hides a real regression. Baselining by rule id accepts every instance of that rule, so a newly broken field under an already-baselined rule slips through. Keep the baseline minimal, add a dated comment for each entry, and schedule its removal rather than letting it grow.

Flaky failures from un-awaited focus or animation. If the error appears with a transition, the scan can catch a mid-animation node. Wait for the settled, visible state and prefer role/label locators over brittle CSS selectors, consistent with the focus-recovery behavior in Focus Management & Keyboard Navigation.

# Fingerprinting Nodes Instead of Baselining Rule Ids

The rule-id baseline in the minimal implementation trades safety for simplicity: accepting color-contrast accepts it on every current and future field. A tighter baseline records a stable fingerprint per violating node, so a brand-new low-contrast field still fails even though the rule is already deferred. axe-core gives you the raw material in each violation’s nodes array — every entry carries a target (a CSS selector path) and the offending html. Hashing those into a set of accepted fingerprints closes the “silently hides a real regression” gap called out above without forcing you to fix all pre-existing debt first.

// tests/a11y/fingerprint.ts
import { createHash } from 'node:crypto';
import type { Result } from 'axe-core';

// A node fingerprint is stable across runs but changes when the
// offending element or its selector path changes — which is exactly
// when we want the gate to re-open on that node.
export function fingerprint(ruleId: string, target: string, html: string): string {
  return createHash('sha1')
    .update(`${ruleId}�${target}�${html.trim()}`)
    .digest('hex')
    .slice(0, 12);
}

// Flatten a violations array into one fingerprint per failing node.
export function fingerprintsOf(violations: Result[]): Set<string> {
  const out = new Set<string>();
  for (const v of violations) {
    for (const node of v.nodes) {
      // target is (string | string[])[]; join the deepest frame path.
      const target = node.target.flat().join(' ');
      out.add(fingerprint(v.id, target, node.html));
    }
  }
  return out;
}

The test then compares fingerprint sets rather than rule-id membership. Any node whose fingerprint is absent from the committed baseline is a regression, even if another node fails the same rule:

import baselineHashes from './axe-baseline-nodes.json';

const found = fingerprintsOf(violations);
const accepted = new Set<string>(baselineHashes);
const regressions = [...found].filter((h) => !accepted.has(h));

expect(regressions, 'un-baselined accessibility violations').toEqual([]);

The trade-off is precision versus churn: because the fingerprint includes the element’s html, an innocent refactor — renaming a class, reordering attributes — invalidates the hash and surfaces the node as “new.” That is usually a feature, since it forces a fresh look, but on a fast-moving form it can produce noise. Include only the target path when you want the baseline to survive markup edits, or both when you want maximum sensitivity. Regenerate the file with a one-off script (node scripts/write-baseline.mjs) rather than editing it by hand, and review the diff in code review like any other snapshot.

# Deduplicating axe Runs Across a Sharded Matrix

Once the suite grows past a handful of forms you will shard Playwright across workers, and a naive setup re-instantiates the full axe rule engine inside every test. AxeBuilder injects and runs the ~4,000-line axe bundle per .analyze() call, so a project with fifty audits pays that cost fifty times. Two levers keep the accessibility job fast without weakening the gate.

First, disable the rules you will never enforce at the source rather than filtering their results after the fact. .disableRules() and a narrow .withTags() stop axe from evaluating checks you do not gate on, which is measurably cheaper than running every WCAG check and discarding most of them. Second, run the audit against the settled error state exactly once per form and assert on the returned object, instead of calling .analyze() repeatedly inside the same test.

// A reusable audit helper keeps configuration identical across shards
// and centralises the tag/rule policy in one place.
import AxeBuilder from '@axe-core/playwright';
import type { Page } from '@playwright/test';

const WCAG_TAGS = ['wcag2a', 'wcag2aa', 'wcag22aa'] as const;

export async function auditForm(page: Page, selector: string) {
  return new AxeBuilder({ page })
    .include(selector)
    .withTags([...WCAG_TAGS])
    // Skipped globally: enforced by a separate design-system check.
    .disableRules(['landmark-one-main', 'region'])
    .analyze();
}

Because each shard runs on its own CI runner, cache the Playwright browser download (shown in the workflow above) and pass --workers=100% locally but a fixed --workers=2 in CI, where hosted runners have two cores. Over-subscribing workers on a two-core runner slows the job through context-switching rather than speeding it up. If total wall-clock still dominates, split the accessibility job from the functional suite so a slow end-to-end test cannot delay the merge-blocking a11y signal.

# Related Guides

• axe-core Accessibility Testing — the scan API, scoping, and reading violations
• Testing & Accessibility — where automated audits sit in the testing pyramid
• Constraint Validation API Deep Dive — the native error DOM these audits inspect

← Back to axe-core Accessibility Testing