attest-standards

Attest-standards is a small CLI application that can build both standard and custom rule-sets.

Installation

To run Attest-standards, you'll need to globally install attest-standards:

npm install -g attest-standards --registry https://agora.dequecloud.com/artifactory/api/npm/attest-virtual/

Custom rulesets overview

A custom ruleset is a data file (in JSON) that is passed to WorldSpace Attest tools to add new rules, or change how existing rules are executed. Custom rulesets are made from two pieces of data:

• New rules and checks, and
• Overriding existing rules and checks

Building and testing a custom ruleset is best done using the attest-standards utility. It can be imported into a custom rules project as a dependency, and will build the attest.json custom ruleset file for you. New rules are placed in a "rules" folder and each .json file should resemble that of an existing rule with respect to structure. The same goes for new checks.

Custom rules directory structure

A typical Attest rulebuilder custom rules directory structure might look like this:

• rules (new rules go in here)
• checks (new checks go in here)
• changes.json (existing rule overrides go in here)
• attest.json (the resulting file returned from the rulebuilder)

Adding new rules or checks is simple - make a new .json file in the respective folder and make sure it has the above properties for either a rule or a check depending on what you’re creating.

changes.json custom rules file

Customizing or overriding existing rules relies on a changes.json file. That might look something like this:

{
  "rules": [
    {
      "id": “accesskeys”,
      "enabled": false
    }
  ],
  "checks": [
    {
      "id": “invalidrole”,
      "metadata": {
        "impact": “minor”,
        "messages": {
          "pass": “Hooray, you used the correct role!”,
          "fail": “Invalid role on this element.”
        }
      }
    }
  ]
}

The above example, how to disable the "accesskeys" rule - it will not run - and in the check, the “impact” was changed to “minor” and updated the pass and fail messaging. You can override any of the properties mentioned above.

Rules and checks

If you are creating a brand new custom rule or check, add your new .json file to the respective folder - rules or checks - and include all of the required properties (see below). These same properties are used in the custom.json file to update properties of existing rules. You can find the existing rules and checks in the aXe-core source files:

• aXe-core rules: https://github.com/dequelabs/axe-core/tree/master/lib/rules
• aXe-core checks: https://github.com/dequelabs/axe-core/tree/master/lib/checks

Rules

Properties

• id (string) This is the name of the rule, for example "accesskeys" which is an existing rule we are overriding.

• enabled (boolean, optional, true by default) Enables or disables a rule. An example of this may be if you’re building a custom ruleset based on Section 508 but you want a few rules from WCAG turned on, you could add them and enable them. The same goes for disabling rules.

• selector (string) The CSS selector for the elements you are targeting. For example, "input, select, button" would target all input, selects, and buttons on the page and run the rule against them. This defaults to * or all elements.

• matches (string, optional) A string of JavaScript code, or, more preferably a string reference to your matches file. In either case your code returns elements that the rule should be run against.

• pageLevel (boolean, optional) Setting this to true would run the rule on the page as a whole, rather than individual elements. It defaults to false.

• tags (array) An array of keywords or tags for your rule.

• metadata (object)

  • help (string) The help text for your rule when it fails. This shows up on the left side of the Attest extension.

  • description (string) Additional context for the failure condition. This appears above the main content area in the Attest extension.

• all (array) Checks that all must pass for this rule to pass. If all of these checks pass, the rule passes. May be empty.

• any (array) Checks where any may pass for this rule to pass. If any of these checks pass, the rule passes. May be empty.

• none (array) Checks where none should pass for this rule to pass. None of these checks should return true. May be empty.

Note that you must have at least one check in all or any or none, or any combination of those. All three of the arrays must not be empty.

Checks

All check properties are required.

• id (string) Much like the rule ID this is the name of the existing check that we are overriding.

• **options *(array) Check options let you customise how the check runs. For example, recently we've added options to allow people to add aria- properties axe doesn't support out of the box. Other things like supported language codes and what color contrast measures to use are also configurable.

• evaluate (string) Similar to matches in the rule, this is the actual JavaScript code that evaluates the element (or node) and returns true or false. This works best when referencing an external check file, such as "my-check.js".

• metadata

  • Impact (string) One of "critical", “severe”, “moderate”, “minor”. Indicates the weight of the failure.

  • messages

    • pass (string) The message for a passing rule.

    • fail (string) The message for a failing rule.

    • incomplete (string) The message for an "undefined" result which will place the rule failure in “Review” as opposed to “Violations”.

    • (string) You can add custom properties here as well.

Example scenarios

You need the severity (impact) changed on a few rules and is using the WCAG base set of rules.

Create a custom ruleset in the Attest Custom Rules project on Bitbucket. Create a changes.json file and add the individual checks with the modified impacts.

Example changes.json file:

{
  rules: [],
  checks: [
    {
      "id": “some-check”,
      “metadata”: {
        "impact": “severe”
      }
    }
  ]
}

You want the help and/or description text changed for some of the rule failures.

Create a custom ruleset in the Attest Custom Rules project on Bitbucket. Create a changes.json file and add the individual rules with their modified metadata.

Example changes.json file:

{
  rules: [
    {
      "id": “some-rule”,
      “metadata”: {
        "help": “Some changed help message.”,
        "Description": “Some changed description.”
      }
    }
  ],
  checks: []
}

You want new rules written that don’t exist in axe-core. No existing rules are modified.

Create a "rules" folder and a “checks” folder. Add your new rule and at least all of the required properties listed above. Your new rule may reference existing checks or you can write your own. You can also mix some existing checks with new checks. All new checks should go in your “checks” folder and include all of the properties listed above for checks.

Example of directory:

rules
  |_ my-new-rule.json
  |_ my-new-rule-matches.js (optional)
checks
  |_ my-new-check.json
  |_ my-new-check.js

You want new rules in addition to some axe-core rule modifications.

Create a "rules" folder and a “checks” folder. Add your new rule and at least all of the required properties listed above. Your new rule may reference existing checks or you can write your own. You can also mix some existing checks with new checks. All new checks should go in your “checks” folder and include all of the properties listed above for checks.

Example of directory:

rules
  |_ my-new-rule.json
  |_ my-new-rule-matches.js (optional)
checks
  |_ my-new-check.json
  |_ my-new-check.js

Example changes.json file:

{
  rules: [
    {
      "id": “some-rule”,
      “metadata”: {
        "help": “Some changed help message.”,
        "Description": “Some changed description.”
      },
      "any": [],
      "all": [
        "some-axe-core-check",
        "another-axe-core-check"
        "my-new-check"
      ],
      "none": []
    }
  ],
  checks: [
    {
      "id": “some-check”,
      “metadata”: {
        "impact": “severe”
      }
    }
  ]
}

In this example we are creating a new rule, a new check, and modifying an existing rule and check. I’ve also added an addition check to the "some-rule" axe-core rule.

Rulebuilder overview

The rulebuilder in attest-standards can produce custom rulesets:

• using built-in rules such as Section 508 or WCAG (A and/or AA), or
• using custom rulesets as specified in changes.json

Using the rulebuilder

There are two ways to use the rulebuilder:

• installed globally and invoked with $ attest-standards
• cloned and referenced with ../attest-standards/bin/generate.js

Rulebuilder command line options/flags

• -h Displays help and available options or flags.

• -t Indicates which tags, space-separated, to build from. For example -t section508 would build a custom ruleset with all of the rules tagged with "section508". You could also use -t wcag2a wcag2aa to get all the rules with WCAG level A and AA.

• -c Points to your changes.json file. The path is relative.

• -d Let’s you specify and output directory for the generated ruleset. Default is the working directory. The path is relative.

• -l If "true" will generate a spreadsheet of the included custom rules. Placed in -d if present or working directory if not.

• -x If "true" (default) will disable all rules not included in the rules property. This is handy if the version of axe-core used to build the rules differs from that used in Attest, which is oftern newer.

Rulebuilder command line use examples

Generating a default ruleset without customizations

• $ attest-standards --<508, wcag2, devmin>

Generating all the default rulesets without customizations

• $ attest-standards build

You may optionally specify a destination with -d and changelogs with -l.

Generating a custom ruleset based on WCAG

• $ attest-standards -t wcag2a wcag2aa -c <dir>

Generating a custom ruleset and a changelog spreadsheet based on Section 508 and specify the output directory

• $ attest-standards -t section508 -c <dir> -d <dir> -l true

Generate a ruleset and include all rules, not just those in the rules property

• $ attest-standards --wcag2a -x false

Advanced topics

Testing

To ensure high quality rules, it is recommended to write integration tests for new rules. These aren’t just important to know that your rule works well today, but to know when things may have broken when upgrading to a newer version of aXe. Additionally you should consider writing unit tests for any evaluate functions you write for new checks, and for any matcher functions you write for rules.

aXe-core common utilities

aXe-core comes with a large collection of utilities to help you write your rules. These are all available on the axe.commons namespace object. There are grouped into 5 categories:

• ARIA: Anything you want to know about well supported ARIA attributes lives here

• Color: Find foreground and background colors and contrast ratios

• DOM: Methods to help you understand DOM Nodes live here

• Tables: Traverse tables and find the relationships between tables

• Text: Whatever you need to know about accessible names and labels of elements

A complete description of the available methods can be found in the aXe-core Commons API documentation: https://github.com/dequelabs/axe-core/blob/develop/doc/API.md

Custom best-practice and experimal rules

Custom rules, just like in aXe-core standard rules, can have best practices and experimental rules. Best practice rules are rules that, like the name says, test for use of (accessibility) best practices. Experimental rules are rules that have not yet ready for prime time, but need to be tested to prove their quality.

Best practice / experimental how-to

1. Add a new best practice or experimental rule In the ‘rules’ directory, add the tag to that rule’s ‘tags’ property.

2. Change an existing WCAG rule to best practice or vise versa: In change.json, add ‘best-practice’ to that rule’s tags array, and remove the ‘wcag2a’ or ‘wcag2aa’ tag. Similarly, you could remove the best-practice tag and add wcag2aa if you wish to make a best practice into a WCAG 2 rule.

3. Remove a best practice or experimental rule In change.json, set that rule’s ‘enabled’ property to ‘false’ and set the tags property to an empty array. This will prevent this rule from ever running.

Shadow DOM rules

Shadow DOM in aXe

With the release of aXe-core 3.0, shadow DOM testing was introduced. This fundamentally changed how aXe-core runs rules and checks. The biggest change is that before executing, aXe core will walk the DOM, looking for any open shadow DOM, and build up the flattened tree (also known as composed tree). This merged all shadow and light DOM trees into a single tree that can be walked and queried. Note that closed DOM trees will not be tested this way.

When running rule selectors, aXe-core will search the flattened tree for matching nodes. For this, it uses the axe.utils.querySelectorAll() function. Because this method looks inside the flattened tree, axe.utils.querySelectorAll will find elements in light and shadow DOM, unlike the browser’s build in querySelectorAll method which only searches the light DOM.

Many aXe-core commons come with a virtual counterpart, which you can recognise because they have "Virtual" at the end of the method name. Instead of taking a DOM node as an argument, these methods must be passed a virtual node - one of the nodes in the flattened tree. These methods travers the flattened tree, allowing you to test across shadow boundaries. All check evaluate functions, and rule matches functions are now passed a virtualNode argument in addition to its node argument, to be used in these *Virtual methods.

Building Shadow DOM rules

When writing rules for aXe 3.0 and beyond, you must be aware that elements you are testing may exist within shadow DOM trees. This changes things in a number of ways.

Parent nodes

The node’s ancestry may cross DOM trees. For instance a common use case is to have a

• is slotted and live in the light DOM. This means the parentNode property will get you the wrong element. Use dom.getComposedParent, and dom.findUpVirtual instead.

  Searching the DOM

  When searching the DOM, for instance for an element with a given ID, you’ll have to think about what tree you want to be searching. There is the light DOM, available on document, there are any number of shadow DOM trees, and there is the flattened tree. ID references, such as those with aria-labelledby do not cross shadow boundaries. So the referenced element must exist in the same tree. To search elements on the same tree, use dom.getRootNode() and than use querySelectorAll. If instead, you are looking for the next heading on a page to see if heading levels were skipped, you will need to search the flattened tree with axe.utils.querySelectorAll().

  Shadow DOM Methods

  aXe-core comes with a number of common utilities. More information about these can be found in the aXe developer guide:

  • https://github.com/dequelabs/axe-core/blob/develop/doc/developer-guide.md