Test methods | a11ied

Methodology

Automated, hybrid, and manual test methods

WCAG is organized into numbered success criteria, such as 4.1.2 (Name, Role, Value) or 1.4.3 (Contrast Minimum). a11ied classifies every criterion into one of three actionable test methods, derived from the official W3C Accessibility Conformance Testing (ACT) rules and Deque axe-core rule tags:

automated Direct rule evaluation. Software algorithms can definitively decide pass or fail by inspecting the DOM, styles, and computed values. For example, 1.4.3 (Contrast) and missing image alt attributes in 1.1.1 (Non-text Content) have automated checks for some requirements. Passing these checks still requires assessment of the remaining criterion requirements.

hybrid Automated execution with human or agent verification. Static analysis can check syntactic prerequisites, but cannot verify whether interactive behavior is correct. For example, 2.4.3 (Focus Order) and 4.1.3 (Status Messages) require driving keyboard navigation or triggering dynamic updates to inspect spoken screen reader announcements.

manual Human review required. Subjective or semantic requirements where no algorithmic heuristic exists. For example, 1.3.3 (Sensory Characteristics) evaluates whether instructions rely solely on shape, size, or sound, and 2.4.6 (Headings and Labels) evaluates whether descriptions are clear and helpful.

Coverage reality

WCAG 2.2 criteria distribution

Teams often assume automated scanners catch the vast majority of accessibility issues. Querying a11ied's bundled WCAG data reveals the real numbers:

a1 wcag criteria --summary

WCAG 2.2 test methods  updated 2026-09-05
  Level      Criteria   Automated  Hybrid     Manual     Unknown
  A          31         0          18         13         0
  AA         24         0          11         13         0
  AAA        31         0          7          24         0
  All        86         0          36         50         0

  28 criteria have an axe rule, 21 have an ACT rule, 19 have both.

At the standard Level AA compliance baseline (55 total criteria across Level A and AA):

Scanner rules cover parts of the criteria; no whole criterion passes on a clean scan alone.
Criterion assessments require interactive keyboard testing, screen reader observation, or semantic review.

This is why a11ied combines axe-core scanning with virtual and native screen reader automation: you can test the hybrid criteria that traditional CI linters miss entirely.

Actionable remediation

Testing hybrid criteria

For hybrid criteria, a11ied provides concrete testing strategies that specify whether an interactive session is required:

a1 wcag 4.1.3 --json

\"testMethod\": {
  \"method\": \"hybrid\",
  \"axeRuleIds\": [],
  \"actRuleIds\": [],
  \"sourceAttribution\": [\"strategy-heuristic\"]
},
\"strategy\": {
  \"preferredEvidenceMode\": \"hybrid\",
  \"procedureIds\": [\"status_message_probe\"],
  \"requiresRealTarget\": true,
  \"notes\": [
    \"Status messages need announcement evidence from assistive technology without forced focus changes.\"
  ]
}

When requiresRealTarget: true is returned, verifying HTML strings in isolation is insufficient. The test requires an active runtime session where you trigger an action (such as saving a form or updating a cart) and assert against the resulting speech queue:

a1 sr start --sr virtual http://localhost:3000/settings
# Trigger update button...
a1 sr transcript --since 5s

AI agent feedback

When an AI coding agent fixes an issue, it inspects strategy.procedureIds to know which tool to invoke (e.g. running sr_action to test keyboard navigation instead of just re-running axe).

Developer toolkit

Querying criteria & rules from the CLI

All WCAG specifications, Understanding documents, techniques, and rule mappings are bundled offline directly in the a11ied package. Queries run instantaneously without network requests:

a1 wcag                                  # Interactive terminal finder
a1 wcag 2.4.3                            # Inspect requirement text, techniques, and test methods
a1 wcag G18                              # Query a specific WCAG technique
a1 wcag rule button-name                 # Map an axe violation to its WCAG criteria and fixes
a1 search "keyboard trap"                # Full-text search across all criteria and techniques
a1 pattern combobox                      # Query ARIA Authoring Practices Guide (APG) specifications

The CLI supports both WCAG 2.1 and WCAG 2.2 (default). Use a1 search when you know what behavior you need to verify but don't know the exact criterion number.

Audit integrity

Making credible accessibility claims

A clean report from an automated scanner means that the elements on your page did not violate the subset of rules that automated algorithms can check. It does not prove that a user who relies on a screen reader or keyboard can successfully navigate your application.

When reporting accessibility status to stakeholders or in compliance audits, distinguish between your evidence sources:

Automated axe scan: Reports mapped scanner results and leaves interaction and semantic requirements pending.

Screen reader test suite: Proves correct focus ordering, widget state announcements, and live region behavior across hybrid criteria.

Manual evaluation: Verifies cognitive, sensory, and content readability requirements that require human judgment.