Continuous AI for Accessibility: Build a Feedback-Driven CI Pipeline with Automated Scans and Developer Feedback

Introduction

GitHub's recent write-up on continuous AI for accessibility shows a practical shift: treat accessibility as a continuous feedback loop, not a one-off checklist. See the original post: Continuous AI for accessibility. This article turns that idea into a developer-focused recipe you can add to CI today: automated scans, lightweight AI-assisted triage, and human-in-the-loop review.

Goals

• Run automated accessibility scans on pull requests and main branches.
• Attach machine-readable results to CI (SARIF / JSON) and create GitHub Check annotations.
• Use a lightweight AI-assisted step to prioritize issues and suggest starter fix snippets.
• Keep humans in the loop for final validation and remediation.

High-level architecture

• CI job runs a headless browser scan (Playwright or Puppeteer + axe-core)
• Scan output is normalized to SARIF/JSON and uploaded as an artifact or Check annotation
• Optional AI step ranks failures and generates suggested code snippets/comments
• Developers review suggestions, accept or discard, and merge with fixes

Why this approach

• Fast feedback: scans run on PRs so issues are found where they matter.
• Prioritization: AI can rank noisy results so teams focus on impactful issues first.
• Auditability: SARIF/JSON artifacts make tracking regressions and trends possible.

Example 1 — Node.js: run Playwright + axe-core and emit JSON

This lightweight script runs a single-page accessibility scan with Playwright and axe-core. Use it in a CI step and save the JSON output as an artifact.

const { chromium } = require('playwright');
const { matchImageSnapshotOptions } = require('@playwright/test');
const AxeBuilder = require('@axe-core/playwright').default;
 
(async () => {
  const url = process.env.TARGET_URL || 'https://example.com';
  const browser = await chromium.launch({ args: ['--no-sandbox'] });
  const page = await browser.newPage();
  await page.goto(url, { waitUntil: 'load' });
 
  const results = await new AxeBuilder({ page }).analyze();
 
  // Normalize and write to stdout (CI job should capture this)
  console.log(JSON.stringify({ url, timestamp: new Date().toISOString(), results }, null, 2));
 
  await browser.close();
})();

Notes:

• Install: npm install playwright @axe-core/playwright.
• Run on a matrix of URLs or on headless builds of your app's routes.
• Save the JSON as a CI artifact for later processing or trend analysis.

Example 2 — Convert results to GitHub Check annotations (JavaScript sketch)

Convert axe results to Check annotations so failures appear inline in the PR. The following sketch shows how to create annotations using the GitHub Checks API via @actions/github. In a real Action, handle pagination, rate limits, and sizes.

const core = require('@actions/core');
const github = require('@actions/github');
 
async function run() {
  const token = core.getInput('github_token');
  const octokit = github.getOctokit(token);
  const { owner, repo } = github.context.repo;
  const head_sha = github.context.sha;
 
  // resultsJson should be the output from the axe run
  const resultsJson = JSON.parse(core.getInput('axe_results'));
 
  const annotations = [];
  for (const violation of resultsJson.results.violations || []) {
    for (const node of violation.nodes) {
      annotations.push({
        path: node.html ? 'inline' : 'unknown',
        start_line: 1,
        end_line: 1,
        annotation_level: 'failure',
        message: `${violation.id}: ${violation.help} — ${node.failureSummary}`
      });
    }
  }
 
  // Create a check run (watch size limits: max 50 annotations per request)
  await octokit.checks.create({
    owner,
    repo,
    name: 'accessibility/axe',
    head_sha,
    status: 'completed',
    conclusion: annotations.length ? 'failure' : 'success',
    output: {
      title: 'axe accessibility results',
      summary: `${annotations.length} issue(s)` ,
      annotations: annotations.slice(0, 50)
    }
  });
}
 
run().catch(err => core.setFailed(err.message));

Important production considerations:

• API annotation size limits — send summaries and attach full JSON as an artifact.
• Map annotations to real file paths/lines where possible (single-page apps may need source maps).

Example 3 — Lightweight AI-assisted triage (pattern)

Instead of trusting raw counts, use a small model or prompt engine that ranks issues by likely impact and suggests a starter fix. The pattern below describes the flow; concrete integration choices vary by team and privacy constraints.

1. Input: axe JSON + page screenshot + component file (if available).
2. Preprocess: group related violations by DOM selector and rule id.
3. Scoring: use simple heuristics (contrast failures + role issues + missing labels = high impact). Optionally call a private/hosted model to rank suggestions.
4. Output: an ordered list of suggested fixes with code snippets and confidence score. Post as a PR comment or check annotation.

Example quick heuristic (Python sketch):

def score_issue(violation):
    score = 0
    if violation['id'] == 'color-contrast':
        score += 5
    if 'aria' not in violation.get('tags', []):
        score += 3
    # Penalize purely cosmetic issues
    if 'experimental' in violation.get('tags', []):
        score -= 2
    return score
 
# Sort violations by score desc
violations_sorted = sorted(results['results']['violations'], key=score_issue, reverse=True)

Keep the AI step optional. For regulated environments or sensitive data, run models on-premises or avoid sending page content outside your control.

Tradeoffs and practical advice

• Noise vs coverage: Automated scanners find many issues but not everything. Use them as a gate for clear regressions, not to block all PRs by default.
• Annotation limits: GitHub checks are useful but limited in size. Post the top N as annotations and link to full artifacts.
• Performance: Scans add CI time. Use caching, sample runs for large suites, or run full scans on main branch only.
• AI suggestions: They accelerate triage but must be reviewed. Treat generated fixes as suggestions, not authoritative changes.

Checklist to add this to your repo

• Add a test script that runs Playwright/Puppeteer + axe and emits JSON.
• Wire that script into your CI (GitHub Actions, GitLab CI, etc.).
• Create a small action/task to convert results into a Check or PR comment.
• Add an optional AI triage step that ranks and suggests fixes; ensure data privacy.
• Monitor artifact trends to spot regressions over time.

Conclusion

Continuous AI for accessibility is about making accessibility a continuous, actionable part of your development workflow. Start small: add automated scans to PRs, attach results as artifacts and annotations, and optionally add an AI-assisted triage step to reduce noise and accelerate fixes. Keep humans in the loop for final decisions and verify fixes with manual testing. This pattern scales: from startups adding quick PR feedback to large orgs tracking accessibility regressions over time.

Further reading: the GitHub engineering post that inspired this workflow: Continuous AI for accessibility.