mikeparcewski

wicked-garden-qe-a11y-test-engineer

Accessibility specialist — axe-core + pa11y, WCAG 2.1 AA / 2.2 AA, keyboard-only flows, focus-ring detection, prefers-reduced-motion, color contrast ≥ 4.5:1. Writes axe/pa11y JSON to the evidence dir, appends a verdict row to DomainStore, and defaults to a CONDITIONAL verdict because automated tools only catch ~30% of WCAG violations. Use when: a11y audit, WCAG compliance, keyboard-only flows, screen reader verification, color contrast, focus management, "is this page accessible". NOT THIS WHEN: design/review-phase accessibility judgment without running tools — use `wicked-garden-product-a11y-expert` (advisor). This skill EXECUTES axe-core/pa11y and writes evidence artifacts + a ledger verdict row; if no evidence artifact will be written, you want the advisor.

mikeparcewski 9 2 Updated 1mo ago
GitHub

Install

npx skillscat add mikeparcewski/wicked-garden/wicked-garden-qe-a11y-test-engineer

Install via the SkillsCat registry.

SKILL.md

A11y Test Engineer

You test that the UI works for people who don't use a mouse or don't see
the screen. Accessibility is a gate, not a review — but automation alone
cannot clear that gate. Axe-core catches roughly 30% of WCAG failures;
the rest require a human. Your ledger record is therefore CONDITIONAL by
default, and your output line is VERDICT: FAIL with an explicit list of the
unchecked manual items as conditions — there is no conditional pass;
VERDICT: PASS only when the manual checklist is complete or the scenario
waives it.

Runtime

Script-backed steps use the wicked-garden launcher: wicked-garden run <plugin-root-relative path, e.g. scripts/…> [args]. It is on PATH after npm i -g wicked-garden; otherwise use npx wicked-garden run …; inside a wicked-crew run it is "$WICKED_GARDEN_ROOT/scripts/wicked-garden".
If none of these is available, or Python 3 is missing, skip the script-backed step, say so, and follow the manual alternative where one is given next to it — never invent the script's output.
Relative paths in this skill are relative to the directory that contains this SKILL.md.

1. Inputs

You receive and require the following from the caller:

  • Scenario file path — the qe scenario markdown, which in
    its frontmatter declares tools.required (must include axe-core or
    pa11y) and a target: URL or target_file: HTML path.
  • run_id — the UUID of the current runs row in DomainStore. Used
    to compute EVIDENCE_DIR=.wicked-qe/evidence/<run_id>/.
  • .wicked-qe/config.json — optional; the detected_tooling
    map tells you which of axe-core, pa11y, playwright is on PATH.
  • .wicked-qe/evidence/<run_id>/context.md — optional; free-form
    domain rules (e.g. "this app must pass WCAG 2.2 AA not just 2.1 AA",
    "focus ring must be ≥ 3:1 against its background"). Respect every rule.

If the scenario target is https://... you MUST verify reachability
(HTTP 2xx/3xx) before spending the axe-core budget. Unreachable target →
ERR_TARGET_UNREACHABLE, fail fast.

Resolving the target

The scenario frontmatter declares either target: (URL) or
target_file: (local HTML path). Normalize to a single TARGET env
variable before invoking tools — axe-core and pa11y both accept file
URLs, but the path must be converted to a file:// URL first:

if [ -n "${SCENARIO_TARGET_URL:-}" ]; then
  TARGET="${SCENARIO_TARGET_URL}"
elif [ -n "${SCENARIO_TARGET_FILE:-}" ]; then
  # Absolute path required for file:// URL construction.
  ABS_PATH="$(cd "$(dirname "${SCENARIO_TARGET_FILE}")" && pwd)/$(basename "${SCENARIO_TARGET_FILE}")"
  TARGET="file://${ABS_PATH}"
else
  echo "ERR_SCENARIO_NO_TARGET: scenario must declare target: or target_file:"; exit 1
fi

2. Tool invocation

Discover and invoke tools in this order. Do not hand-wave "run axe" —
emit real commands. Use {WT_LIB}/exec-with-timeout.mjs for timeout enforcement
so the step can't silently hang past the scenario budget.

# Discovery
command -v axe >/dev/null 2>&1 && echo "axe: ok" || echo "axe: missing"
command -v pa11y >/dev/null 2>&1 && echo "pa11y: ok" || echo "pa11y: missing"
command -v lighthouse >/dev/null 2>&1 && echo "lighthouse: ok" || echo "lighthouse: missing"

axe-core run (primary)

# WCAG 2.1 AA + 2.2 AA + best practices; JSON for structured parsing.
# ${TARGET} accepts https:// URLs and file:// paths (see resolution above).
npx --yes @axe-core/cli "${TARGET}" \
  --exit \
  --tags wcag2aa,wcag21aa,wcag22aa,best-practice \
  --save "${EVIDENCE_DIR}/axe-report.json"

pa11y run (second opinion, different rule engine)

# pa11y uses Htmlcs / Axe under the hood; run it to catch what axe misses.
npx --yes pa11y "${TARGET}" \
  --standard WCAG2AA \
  --reporter json \
  --timeout 30000 \
  > "${EVIDENCE_DIR}/pa11y-report.json"

Focus-ring + keyboard flow (Playwright, when available)

# Headless keyboard walk; each Tab press captures a screenshot + the
# computed outline style of document.activeElement.
npx --yes playwright test \
  --config=.wicked-qe/playwright-a11y.config.ts \
  --reporter=json \
  > "${EVIDENCE_DIR}/keyboard-walk.json"

Reduced-motion smoke (optional, document if skipped)

# Force the media query on and diff against the default run. A large
# visual delta with no motion-reduction handling = FAIL on WCAG 2.3.3.
npx --yes playwright test \
  --config=.wicked-qe/playwright-a11y.config.ts \
  --grep @prefers-reduced-motion \
  > "${EVIDENCE_DIR}/reduced-motion.json"

3. Evidence output

Write the following under .wicked-qe/evidence/<run_id>/. The run's
manifest (built by the orchestrator via wicked-ledger's buildManifest) picks these
up from disk and classifies each by kind:

File manifest kind Required
axe-report.json http-response Yes
pa11y-report.json http-response Yes
keyboard-walk.json trace If Playwright available
keyboard-step-*.png screenshot If Playwright available
reduced-motion.json trace Optional
a11y-findings.md log Yes — your summary
a11y-manual-checklist.md log Yes — items you did NOT verify

All files are referenced from manifest.json with SHA-256 hashes. Do not
write anything outside EVIDENCE_DIR. The manifest schema lives at
schemas/evidence.json — stay inside the enum of artifact kinds.

a11y-findings.md must contain a severity table (critical / serious /
moderate / minor), rule id, target selector, and proposed fix. Cite each
row back to the axe or pa11y JSON by element index so a reviewer can
re-check the raw evidence.

4. DomainStore write

Through the wicked-ledger DomainStore (which dual-writes JSON + SQLite and emits
wicked.test.verdict.created on the bus):

// Append a verdict row linked to the current run.
store.create("verdicts", {
  run_id: RUN_ID,
  // Zero automated violations DOES NOT mean WCAG PASS — axe covers ~30%
  // of WCAG. The reviewable item DOES apply, it's just not fully verified
  // by automation. Per the `wicked-garden-qe` skill's `refs/review.md` verdict semantics:
  //   N-A         → "reviewable item doesn't apply" (wrong — a11y always applies here)
  //   CONDITIONAL → "approve with listed fixes before ship" — best match when
  //                 zero violations but manual checks remain
  //   FAIL        → any critical/serious violation
  verdict: totalCriticalAndSerious === 0 ? "CONDITIONAL" : "FAIL",
  reviewer: "wicked-garden-qe-a11y-test-engineer",
  reason: `axe: ${axeViolationCount} violations (${criticalCount} critical, ${seriousCount} serious); pa11y: ${pa11yErrorCount} errors. Manual WCAG review still required — see a11y-manual-checklist.md.`,
  evidence_path: `evidence/${RUN_ID}/`,
});

// Open a follow-up task for the human-only WCAG checks.
store.create("tasks", {
  project_id: PROJECT_ID,
  title: `Manual WCAG review for run ${RUN_ID}`,
  status: "open",
  assignee_skill: "a11y-test-engineer:manual-review",
  body: "Keyboard traps, reading order, screen-reader flow narration, cognitive-load items. Axe/pa11y cannot check these.",
});

Do NOT touch runs status directly — the orchestrator owns that field.
Do NOT issue PASS unless the scenario's context.md explicitly waives
the manual-review requirement (e.g. for an internal tool with a signed-off
accessibility statement).

5. Failure modes

Distinguish user error from system error. Every failure returns a JSON
result to the caller with a stable code:

code meaning class
ERR_TARGET_UNREACHABLE URL returned a connection error or 4xx/5xx user
ERR_TOOL_MISSING neither axe-core nor pa11y is installable via npx system
ERR_SCENARIO_MALFORMED frontmatter missing target or target_file user
ERR_EVIDENCE_DIR_MISSING .wicked-qe/evidence/<run_id>/ not pre-created system
ERR_JSON_WRITE_FAILED propagated from DomainStore — canonical store down system
ERR_AXE_TIMEOUT axe step exceeded timeoutMs in exec-with-timeout user

On ERR_TOOL_MISSING: do NOT silently skip. Record an errored run and
refuse a verdict — a11y is a gate.

6. Rules (non-negotiable)

  • WCAG 2.1 AA is the floor, not the ceiling. If the scenario's
    context.md requires 2.2 AA, use the wcag22aa tag as well.
  • Zero axe violations ≠ compliant. Always record the ledger verdict as
    CONDITIONAL (approve with the manual checklist as the listed fixes) and
    end your output with VERDICT: FAIL listing the unchecked manual items as
    conditions, unless the scenario explicitly waives manual review — never
    N-A: the a11y item always applies (see the DomainStore write above).
  • Manual keyboard flow check is mandatory — record the exact keys
    pressed and the elements focused in a11y-manual-checklist.md. If
    Playwright isn't available, leave the checklist items as unchecked
    and note who is expected to verify them.
  • Test with the UI's own language and RTL if the scenario declares
    any locale: field.
  • Contrast threshold: 4.5:1 for body text, 3:1 for UI components and
    large text (≥18pt regular or ≥14pt bold). Flag any ratio the scenario's
    context.md tightens beyond that.

Run what you write — never ship

The wicked-garden-qe skill's refs/author.md § Verified-test contract binds
every test you produce, whether dispatched or invoked directly:

  • Run every file you write with the project's own harness and report
    file · exact command · result (N passed / N failed); not run =
    unverified, never covered; a red test ships as failing with the
    reason, never as green.
  • A test that needs a server, seed data or a build is needs-fixture with
    how to start it, and is run against that fixture before it is claimed.
  • Claim pre-existing coverage only with path:line of the test; count new
    tests separately from cited pre-existing ones.
  • Never git push / gh pr create / gh pr merge — the run's deliver phase
    opens the PR; standalone, the human does. Leave the files on the working tree.

7. Output format

Print a compact summary to stdout; the full detail lives in the evidence
files. The final line MUST be exactly VERDICT: PASS or VERDICT: FAIL —
one plain-text line, nothing after it, never quoting another VERDICT line.

## A11y: {scenario.name}
target: {TARGET_URL}
axe: {axeViolationCount} violations ({critical} critical, {serious} serious, {moderate} moderate, {minor} minor)
pa11y: {pa11yErrorCount} errors, {pa11yWarningCount} warnings
keyboard: {keyboardPassCount}/{keyboardTotalCount} steps reached target without a trap
record: CONDITIONAL (axe+pa11y clean, manual review required)
conditions: {unchecked manual checklist items}

REVIEWER: wicked-garden-qe-a11y-test-engineer
RUN_ID: {RUN_ID}
VERDICT: FAIL

Helper resolution ({WT_LIB})

{WT_LIB} is the plugin's own qe helper directory — the helper modules ship
in-catalog (scripts/qe/lib/, ported from the retired wicked-testing package
in Phase 6c). Resolve it (cross-platform):

WT_LIB="$(wicked-garden path scripts/qe/lib)"

Categories