expect-axe-playwright

expect-axe-playwright

Expect matchers to perform Axe accessibility tests in your Playwright tests.

Installation

npm

npm install expect-axe-playwright

Yarn

yarn add expect-axe-playwright

Usage

// playwright.config.ts
import { expect } from '@playwright/test'
import matchers from 'expect-axe-playwright'

expect.extend(matchers)

Why do I need it?

This project was inspired by axe-playwright which did a great job of integrating the axe-core library with some simple wrapper functions. However, the API is not as elegant for testing as would be preferred. That's where expect-axe-playwright comes to the rescue with the following features.

• Direct integration with the expect API for simplicity and better error messaging.
• Automatic Axe script injection.
• Auto-retry until timeout.
• Works with pages, frames, and locators.
• HTML report with full violation details.
• Project-level option configuration.

Here are a few examples:

await expect(page).toPassAxe() // Page
await expect(page.locator('#foo')).toPassAxe() // Locator
await expect(page.frameLocator('iframe')).toPassAxe() // Frame locator

API Documentation

toPassAxe

This function checks if a given page, frame, or element handle passes a set of accessibility checks.

You can test the entire page:

await expect(page).toPassAxe()

Or pass a locator to test part of the page:

await expect(page.locator('#my-element')).toPassAxe()

You can also pass an Axe results object to the matcher:

import { waitForAxeResults } from 'expect-axe-playwright'

test('should pass common accessibility checks', async ({ page }) => {
  const { results } = await waitForAxeResults(page)
  await expect(results).toPassAxe()
})

Word of Caution: Limitations to Accessibility Tests

toPassAxe() !== toBeAccessible()

It's important to keep in mind that if your page passes the set of accessibility checks that you've configured for Axe, that does not mean that your page is free of all accessibility barriers.

In fact, automated testing can only catch a fraction of the most common kinds of accessibility errors.

Accessibility is analogous in ways to security. Imagine the following code:

expect(myApp).toBeSecure()

It's very hard to say that anything is secure because you never know when someone is going to uncover a security vulnerability in your code. Similarly, it's very hard to say that anything you've built is totally accessible because you never know when somebody will uncover a barrier you didn't know was there.

Furthermore, of the commonly known accessibility barriers, only some can be found through automated testing, which is then further subject to the effectiveness of the checker being used. A 2017 study on the effectiveness of automated accessibility testing tools by the UK's Government Digital Service confirms this.

To echo jest-axe, tools like Axe are similar to code linters and spell checkers: they can find common issues but cannot guarantee that what you build works for users.

You'll also need to:

• test your interface with the assistive technologies that real users use (see also WebAIM's survey results).
• include disabled people in user research.

Axe run options

You can configure options that should be passed to aXe at the project or assertion level.

To configure a single assertion to use a different set of options, pass an object with the desired arguments to the matcher.

await expect(page).toPassAxe({
  rules: {
    'color-contrast': { enabled: false },
  },
})

To configure the entire project to use a different set of options, specify options in use.axeOptions in your Playwright config file.

// playwright.config.ts
import { PlaywrightTestConfig } from '@playwright/test'

const config: PlaywrightTestConfig = {
  use: {
    axeOptions: {
      rules: {
        'color-contrast': { enabled: false },
      },
    },
  },
}

export default config

Report options

You can configure options that should be passed to the aXe HTML reporter at the assertion level.

await expect(page.locator('#my-element')).toPassAxe({
  filename: 'my-report.html',
})

This is particularly useful if you need to produce multiple aXe reports within the same test as it would otherwise keep replacing the same report every time you run the assertion.

Thanks

• axe-playwright for the inspiration and groundwork laid for using Axe with Playwright.