Kavin Arvind

Topic: Automated Accessibility testing in Devops Environment

Abstract: Automated Accessibility testing in Devops Environment
Importance of Automated Accessibility testing
• When designing and upgrading CI / CD pipelines, it is important to consider functional testing and integration test from a product quality perspective. The introduction of non-functional testing for accessibility testing in the CI CD pipeline offers significant advantages. When it comes to testing security, performance and accessibility, we can be sure that your application will always meet the highest standards.
• Integrating accessibility tests into CICD pipelines is easy, inexpensive and offers us many advantages. The sooner you detect and fix accessibility issues, will not impact the users with disabilities and avoid company’s risking against any legal problems. It also helps in creating an overall high quality product. After all, accessibility rules work just fine for all end users.
• 15% of world population have some sort of disability and creating applications they don't use is like ignoring a significant portion of the market. Creating websites and applications accessible to everyone is an essential part of design and development. However, ensuring digital accessibility has become an arduous and difficult task.

Challenges with Manual Accessibility testing
• Testing accessibility manually can be a challenge for testers, especially if they are not disabled. To them, the look of a website seems good when viewed, but it is difficult to understand. Yes, any tester can use specific instructions or details, but this type of test is still difficult for most people.
• Understanding and adopting Web Content Accessibility Guidelines (WCAG) takes time, and the hurdle becomes even greater when working on group projects.
• It is important that each member to be informed about and committed to web accessibility (a11y) implementation, which is not always apparent to the presentation of the site or app.
• When working against a deadline, regression testing every page simply isn’t feasible.
Automated Accessibility testing

Since manual testing is difficult and complex, we can try automated accessibility tests which is very reliable. Most of tools are open source, which allows automated accessibility testing and integration with devops. Running accessibility test automation rules during continuous integration prevents common accessibility issues

Automated Accessibility testing - CICD Strategy
It is best to consider Accessibility testing in the early phase of the project, so it will help with early testing in development phase to detect and correct the accessibility conformance issues. The outcome of this strategy is to catch bugs as early as possible to ensure high quality roll outs.
There are many open source tools available that can be integrated with CICD , in this white paper we are going to see about pa11y - a popular open source accessibility checker. Pa11y tests for WCAG 2.0 standards with configuration available to specify A, AA, or AAA.

Prerequisites
• Front End task runner (Node js)
• Continuous Integration (CI) tool
• Accessibility testing tool (Pa11y)
• Content (website(s))

1. Integration of pa11y with Jenkins pipeline:
Pa11y is typically added to projects as a Node.js dependency (e.g. using NVM / NPM).
With Pa11y CI library, we can integrate it with any continuous integration pipeline
• Install the necessary npm packages
• Set up Pa11y CI configuration
• Create sitemap
• Add custom scripts
• Add custom git hooks

2. Introduce a step in our Jenkins pipeline
Once a feature has been built and made accessible, Pa11y should automatically test it every time there is a change in code. In other words, every pull request should trigger a CI build and that build should run Pa11y. This helps ensure if a change is being made that might impact the accessibility of a feature, the build will fail, and you will be notified.

Advantages:
• The main benefit of automated accessibility testing in CICD, helps to detect bugs caused by code violations. This saves development time and covers common flaws that are often overlooked.
• Another benefit is the ability to protect against regressions. In an environment where there are many prints a day, where it is not feasible to accessibility test each new print (release) with manual testing, this becomes an indispensable tool to test with a11y standards.