Automating an Accessibility Evaluation Report using Deque Axe and Selenium Webdriver
TLDR; The Deque Axe Core library works with WebDriver and Playwright to generate an automated accessibility report.
Accessibility Testing
Accessibility Testing is important to ensure web applications and mobile applications are usable by anyone. Countries have laws to mandate compliance. But it is a specialized skill set, so most of us don’t actually test for Accessibility we run a few tools and use that for our compliance.
In this post I’ll describe a way of generating an accessibility report using the Deque Axe library driven by Selenium WebDriver and Java.
This post was triggered by reading the Accessibility Testing guide in the Playwright documentation which describes use of the Axe Core Playwright library. Everything in that guide is available to Selenium WebDriver users via the Axe Core Selenium library. The Playwright guide, linked above, is worth reading because it covers more aspects of the Deque Axe API than I do here, and everything that the guide describes is also applicable to WebDriver.
I do not intend to cover Accessibility Testing in detail, that is far bigger topic. Here are some accessibility resources if that’s what you’re looking for:
- Web Content Accessibility Guidelines (WCAG)
- Web Accessibility Directive EN 301 549
- Web Accessibility Directive EN 301 549 differences from WCAG
I particularly like this comment from Steve Green (Accessibility Testing expert) from Test Partners on the Ministry of Testing Club forums. In the post Steve cautions about trusting tools too much and he emphasises the depth of testing required for a proper Accessibility Testing process. read it here
Deque Axe
Deque Axe is a set of Accessibility Testing Tools:
There are browser extensions, CLI tools and libraries that we can hook into our automated execution.
Deque provide the axe core for maven easy to use Java library for Playwright and Selenium WebDriver. They also have integrations for Nightwatch and other TypeScript libraries.
Video Overview
Add to Dependencies
Deque Core for Maven requires a single maven dependency for WebDriver via the Axe Core Selenium.
e.g.
com.deque.html.axe-core
selenium
4.10.2
Simple Integration with WebDriver
To find if there are any issues on a page I could create a WebDriver test:
@Test
void expectPageToHaveAccessibilityIssues() {
driver.get(
"https://testpages.eviltester.com/styled/basic-web-page-test.html"
);
Results accessibilityScanResults = new AxeBuilder().analyze(driver);
assertFalse(
accessibilityScanResults.violationFree(),
"You should have accessibility issues on the page"
);
}
This would give me a Yes/No evaluation. This test should fail because there are known violations on the page.
Seeing the Violations
The axe library has an easy to use reporting class which can make it simple to see what issues are present.
List violations = accessibilityScanResults.getViolations();
AxeReporter.getReadableAxeResults("Default", driver, violations);
System.out.println(AxeReporter.getAxeResultString());
The above would print a human readable report to the console with information like the following (I formatted it a little for the web). The benefit of the report is that it links off to the dequeuniversity.com site where you can learn about the violation.
ACCESSIBILITY CHECK
DEFAULT check for: https://testpages.eviltester.com/styled/basic-web-page-test.html
Found 3 items
1: element must have a lang attribute
Description: Ensure every HTML document has a lang attribute
Help URL: https://dequeuniversity.com/rules/axe/4.10/html-has-lang?application=axeAPI
- Impact: serious
- Tags: cat.language, wcag2a, wcag311, TTv5, TT11.a, EN-301-549, EN-9.3.1.1, ACT
- HTML element:
- Selector: [html]
2: Document should have one main landmark
Description: Ensure the document has a main landmark
Help URL: https://dequeuniversity.com/rules/axe/4.10/landmark-one-main?application=axeAPI
- Impact: moderate
- Tags: cat.semantics, best-practice
- HTML element:
- Selector: [html]
3: All page content should be contained by landmarks
Description: Ensure all page content is contained by landmarks
Help URL: https://dequeuniversity.com/rules/axe/4.10/region?application=axeAPI
- Impact: moderate
- Tags: cat.keyboard, best-practice
- HTML element:
- Selector: [.page-body]
Limiting Page Scope of the Audit
The previous code used a default AxeBuilder but it is possible to include and exclude page elements.
AxeBuilder filteredAxeScope = new AxeBuilder().include(".page-body");
This would only scan elements with a class page-body.
Limiting WCAG Scope of the Audit
The WCAG standard lists a lot of sections and rules to comply with.
It is possible to limit WCAG to a subset of rules using tags.
The full list of tags is in the Deque API documentation:
AxeBuilder filteredAxeScope = new AxeBuilder().withTags(List.of("wcag2a"));
An Example
I added a project on Github with Deque and WebDriver.
There is a test using the a UK Government provided set of test pages.
There are a lot of child pages available from this root page, if you are interested in automating Accessibility then I’d suggest using the alphagov site as a start point.
@Test
void expectPageToHaveAccessibilityIssues(){
WebDriver driver = new ChromeDriver();
driver.get(
"https://alphagov.github.io/accessibility-tool-audit/test-cases.html"
);
Results accessibilityScanResults = new AxeBuilder().analyze(driver);
List violations = accessibilityScanResults.getViolations();
AxeReporter.getReadableAxeResults("Default", driver, violations);
System.out.println(AxeReporter.getAxeResultString());
assertFalse(
accessibilityScanResults.violationFree(),
"You should have accessibility issues on the page"
);
driver.quit();
}
Experiment And Learn More
The Deque Axe tool has a lot of resources on their site.
Of particular interest are this set of JavaScript patterns to see how accessibility is impacted.
And this set of accessible components:
These might prove useful for automating against, and comparing with your own applications.
I’m not an accessibility expert and the best short list of tools I could find was from Steve Green in his answer on the Ministry of Testing forums.
Also linked from the forum answers was Aaron Flynn’s article on Accessibility Testing experiences.
The above article is where I found a link to the interesting 2017 blog post on the UK government site listing the results of using multiple tools for accessibility.
That post, may be most valuable for the set of test pages they created to evaluate tools against, this could be a useful page for testing and learning accessibility. I don’t know of any other sites designed for testing accessibility:
- alphagov.github.io/accessibility-tool-audit/test-cases.html
- github.com/alphagov/accessibility-tool-audit
Accessibility Testing is not an area I am qualified to comment on heavily, which is why I would treat any of the results from the Axe library as a bare minimum.
During web searching I found that the w3.org site has a huge list of tools - too large really.
A large, large, list of tools for checking accessibility both free and paid tools.
It will take quite a long time to work through this list but if you are serious about accessibility testing then this is probably where you would start. Pick a tool, try it, learn what it did, study accessibility.
I sampled a few tools and I’d probably end up using the following.
- the AXE report from this post.
- Chrome Developer tool functionality:
Conclusion
I am not an Accessibility Testing Expert, I run a few tools to help reduce compliance problems.
The Deque Axe Core Maven Library provides a free and easy way to access the Deque Axe tooling from Java (WebDriver and Playwright).
There are a whole host of tools out there which can help.
https://www.patreon.com/c/eviltester">Join our Patreon from as little as $1 a month for early access to videos, ad-free videos, free e-books and courses, and lots of exclusive content.