A11y Beyond the Theory: Integrating Accessibility Testing Into Your Workflow

Rate this content

A practical look at automating core accessibility testing and integrating it into your workflow.

24 min
15 Nov, 2023

AI Generated Video Summary

Ntandala Kengose, a software developer, emphasizes the importance of accessibility in software development and the responsibility it carries. The Web Content Accessibility Guidelines (WCAG) provide technical guidelines for making web content more accessible. Ntandala shares various accessibility testing tools and highlights the need for automation in testing. Tools like Pelly CI and GitHub Actions can be used for automated accessibility testing and CI integration. The X-Accessibility Ginter and Husky are tools that provide insights and ensure accessibility in development.

1. Introduction to Ntandala Kengose

Short description:

Hello, everyone. My name is Ntandala Kengose. I am a software developer from Johannesburg, South Africa. I work for PbD, a global software development firm. I'm part of the ATC unit, responsible for specialized consulting and training. Follow me on Twitter at unlikely underscore.

Hello, everyone. My name is Ntandala Kengose. I am a software developer, among many other people. I am giving this talk from the beautiful city of Johannesburg, where I was born, raised, and live till today.

Now, if you have no idea where Johannesburg is, it is a big city, not the capital, but definitely the economic hub of the beautiful country known as South Africa. And if you're not entirely sure where South Africa is, well, it's simple. It's in the to look at it. We are at the southern tip of beautiful continent of Africa. Now, this should be showing you that when they said that some of the most difficult things in computer science is naming things, they definitely didn't think about South Africans, because clearly we're really good at this.

As I said, I am a software engineer, and I work for a company called PbD. And again, because we're good at naming things, PbD stands for the founders. Now, PbD started about thirty nine years ago, right here in South Africa by three engineers. And we've now grown to be a global bespoke software development firm with probably about one thousand two hundred professionals spread across seven cities all over the world. We deliver bespoke software solutions into a number of sectors. But my job is slightly different from everyone else at PbD. That's because I work in a unit we call ATC. In paper we're responsible for a number of things, such as what we call specialized consulting. We are responsible for training PbD's 1000 plus employees. We're responsible for also doing community engagements and a whole bunch of other things that are quite interesting. I love community. And to fuel this passion, I also am involved with number of meetups around Johannesburg.

None of this matters. The most important thing for you to know about me is that my Twitter handle, X, is at unlikely underscore. Please feel free to follow me, let me know what you think of this talk. Let me dive straight into it. I turned 32 days before the recording of this video. Now, with this new age, I've realized that I need to start looking after myself a bit better. I need to get back into fitness and try and look after my health and what I eat. So a friend of mine strongly suggested that I try the gym. Now, there's a couple of problems with the gym.

2. The Importance of Accessibility

Short description:

I injured my knee while playing soccer and it made me realize the importance of accessibility. I experienced a temporary disability and struggled with inaccessible designs. The responsibility for accessibility lies with everyone involved in the software development life cycle. It's not enough for solutions to work on one machine. Accessibility affects the bottom line.

That's because it can be quite intimidating when you walk in and you see all of the heavy equipment and some of the things that people are doing can just be daunting. So I decided to go back to things that I actually enjoy, which is sports, two sports in particular, rather, namely football and soccer. And no, I don't mean it like this, I mean it more like this because I seem to be really, really prone to injuries.

I play two sports, one where the players are known for faking their injuries, and the other way the riders are really known for trying to avoid their injuries and getting back onto their horse. So take a wild guess as to which one of these two sports ended up or resulted in me having this knee brace on my knee for over three months. It's soccer. I had a minor incident, and I ended up damaging my knee.

Now, something interesting started happening after this injury. I started being very, very grumpy. My friends said it was just old age creeping in, but I realized that something else was happening. I was starting to experience the world like never before. I was experiencing it as someone who couldn't walk as long as I normally do, and I realized that I was actually experiencing a temporary disability and that this was making inaccessible designs around me much clearer. And all of those things were now starting to affect and frustrate me.

And the first thing that really, really, really made me angry was the shopping mall. I live about a block away from one, and I've always thought that it had parking all around it and was a great experience to go to. I hated it because when people think accessibility, all they think about is ramps. And I found these ramps everywhere, and I don't know if you've ever tried, but walking with crutches on a ramp is really, really difficult. And I struggled a lot. I was so furious and I did what every normal person would do, try and figure out whose responsibility is it. And now that I was faced with my own physical limitations and trying to figure out who to blame, it dawned on me. I started reflecting about whose responsibility it is to ensure that the solutions we build are as accessible to as many people as possible.

And to answer this, I think we just need a quick, common working definition of what we mean by accessibility. Well, firstly, A11Y is a numeronym where the 11 stands for, it represents the 11 letters in the word accessibility. And the major dictionaries really gravitate towards this general definition, where they define accessibility to be the quality of being easily reached, entered or used by people who have a disability. If we bring our focus much closer to home, we see that web accessibility means that websites, tools and technologies are designed and developed so that people with disabilities can use them. More specifically, that they follow the PAW principles, which means that people can perceive them, understand, navigate and interact with the web that we build.

So to simply answer the question, the answer is that it's everyone's responsibility, everyone that is involved in the software development life cycle. We have long agreed that it works on my machine is simply not enough. So I've always wondered why we're so comfortable with shipping solutions that work for some people and not necessarily others. And every time I've been engaged in conversations around accessibility, it's often treated as a nice to have, but what we don't realize is that it actually affects the bottom line.

3. Web Content Accessibility Guidelines

Short description:

Inaccessibility costs businesses significant amounts of money. The Web Content Accessibility Guidelines (WCAG) are a set of technical guidelines aimed at making web content more accessible. They have 13 guidelines that fall under four core principles. There are three levels of conformance: A, AA, and AAA. Common accessibility challenges are often the easiest to fix.

Inaccessibility costs businesses significant amounts of money. And I mean, we're also seeing it becoming a legal requirement all over the world. So the W3C has put together something they called the Web Content Accessibility Guidelines or WCAG for short. Now this is a set of technical guidelines that are aimed at making web content more accessible to people with disabilities. It effectively has 13 guidelines, as of 2.1. And the guidelines fall under four core principles, meaning that they are aimed at making sure that things are perceivable, operable, understandable, and of course robust.

Having guidelines and principles is one thing, we also need to be able to measure the success. We need to be able to describe just how well we conform to these guidelines. And for this, we've got really, really simple three levels of conformance. Namely, A, AA, and AAA, which range from simple and very common things such as missing alternative text, keyboard accessibility, all the way to the other end of the complex spectrum, where those issues are more concerned about the content and comprehension of our users. The interesting thing is that if you take a look at some of the most common challenges, they are often the easiest to fix. So things like the lack of alternative text, inaccessible forms and things like poor color contrast can actually be looked at quite programmatically. Even the lack of keyboard accessibility is still one of the most common accessibility issues that are found on the web. And, I mean, we've long seen that keyboard shortcuts or the use of those shortcuts can improve productivity quite significantly.

4. Accessibility Testing Tools

Short description:

I recently had a conversation with a senior colleague who shared a story about a client who was upset when accessibility testing caused delays. This made me realize the need to equip people with tools and practical knowledge. I started searching for accessibility testing tools and found the Web Accessibility Evaluation Tool List, which offers a wide range of filters. In this talk, I'll share some of my favorite accessibility testing tools, starting with Lighthouse and Wave.

I recently had a conversation with one of my senior colleagues and he told me about how we once had a client and part of our service offering was that we guaranteed this public service client that we would be able to have someone test the solution to make sure that it's accessible. He told me a story about how because they had this one person, a person who was blind, use the system. A lot of the time when it failed, the client would actually be quite upset about the amount of time that we were adding to the process. And I realized something crucial about what he said.

For such a massive software solution, they had a single person and just said to them, tell us if you can use this. They often released a lot of these solutions without passing quality assurance measures because they were chasing deadlines. I then realized that we actually need to equip people that are involved with the tools that are there to put the tools that can educate them rather of what needs to be done and how to go about implementing these things. But where do we even start with this? I'm a practical person so I don't believe in simply sending people off to a training session or a workshop. I think we need to do more and it needs to be in practice. And for this, practicality is key.

So I started digging around for a couple of solutions, and effectively, this mission was to find accessibility testing tools for everyone that I work with. I started with some of my favorite people as a developer. QA engineers, test analysts. They come in many forms, shape, and names, but we all know that they are a developer's worst nightmare. Now, when I talk about QA engineers, I'm talking about manual testers, for instance. What I found is that there's something called the Web Accessibility Evaluation Tool List. I absolutely love this website, because on the left hand pane it also gives you an ability to filter as to what type of tools there are based on either the guidelines that you want to follow, the legal requirement, or the type of accessibility concern that you're trying to address. Now this tools list has more than 100 tools, because we are still in the web and we get a tool every minute, but the filters are really awesome.

So effectively, the rest of this talk is going to be me telling you about a list of some of my favourite accessibility testing tools. But to do this, we need a website. Now I thought of using the React US website, but then I realised that I really want them to invite me back to this conference next time. So I found this test website, it's called the Medical Mysteries Club. It's built by the Chrome DevTools team and they built it with a disclaimer saying that this page was deliberately made inaccessible for the purposes of the demo, which is quite cool. You can check up the site, there's a code pen for it and here I'm running it in debug mode. Speaking of the Chrome team, my first tool is Lighthouse. Lighthouse is a really, really great tool and it comes shipped with the Chrome browser. And this tool is quite common, you simply need to do a click analyse test in your dev tools and it should be able to give you a good breakdown of some of the accessibility issues. Now, Lighthouse is often used for other things as well such as SEO and performance, and so I won't dive into it a lot because I understand that a lot of people would have seen it. I'll show you similar tools instead. The first one is called Wave, which stands for the Web Accessibility Evaluation Tool.

5. Accessibility Testing Tools Continued

Short description:

Now, this is a browser extension that annotates the screen, showing you issues with icons. XDevTools is a similar paid tool. For project managers, Pally is an open-source tool that provides overviews of accessibility issues. It can be used in the command line or in a node application. Peli Dashboard allows you to track progress and transitions over time. Intercepting accessibility issues at the developer level is crucial, and automation is the solution.

Now, this is a browser extension and once you've added it, it actually annotates the screen that you have in front of you, so it will put all of these tiny icons to actually show you what these issues are. What I like about this is that when you go into detail about what actually is wrong in there, it's able to show you things like such as the low color contrast and even help you figure out what color is best at that point.

Another tool that's quite similar to Lighthouse is called XDevTools. Now, XDevTools also is added in your Developer Tools as a tab. You also just click Analyze, and it will give you a breakdown of all of the issues that are there. One thing to note though is that this is paid, and by the time I started building up this talk, I had already run out of my trial license. The important thing for me here is that I wanted something that I can use for everyone that I work with.

I looked around and I found something for the project managers. When you're managing a project, you're not necessarily interested in the very minute details, but rather overviews of things that are happening. For this, I found one of my favorite tools called Pally, the automated accessibility testing file. Now, Pally is an open-source tool that helps developers and testers identify these accessibility issues on web pages and applications. It can be found on npm, of course. When we take a look at this, we see that it can run in the command line or can be used in your own solution in a node application. What that looks like is that you would simply import it, and you would provide it with the URL. It would go and crawl that web page and return those results to you. You can then do as you wish.

Here, if we go and replace our testing website with the CodePen for the Medical Mysteries Club, we see that if we go and print out those results, this is what it start looking like. What I like about this is that it tells you which principle or which guidelines you are violating. It's able to show you a human-readable message. It gives you the context of where the issue actually came from, and even the HTML selector. Now, this is really interesting. I realized that I'm onto something here, but I'm a web developer. As soon as I start thinking that I'm building something very useful, I know that I need to stop because chances are someone else has probably built something similar to me. For this, I found Peli Dashboard. Now this Peli Dashboard, you can configure the URLs that you want to go and take a look at, so that you can see your transitions over time. This is absolutely great if you are making an already existing application more accessible because you can track your progress and the progress of your team over time. But of course, if the only time you identify accessibility issues is in production, then it's really, really too late. You generally want to intercept these problems at the developer level. You want to guide your developers to solve these problems while they are building. But how do you do this? Well, we automate this.

6. Automated Accessibility Testing and CI Integration

Short description:

Automating accessibility testing and blocking pull requests if they fail is crucial. To achieve this, we can use tools like Pelly CI and GitHub Actions. Pelly CI is an accessibility test runner focused on continuous integration, while GitHub Actions is a widely used CI-CD pipeline. In terms of testing setup, Mocha, Jasmine, and Jest are popular tools for test running, assertion, and browser rendering. Jest, in particular, is a delightful JavaScript testing framework that simplifies the process.

Automates the testing portion of this. Of course, developers are new to testing. We all write code that we've tested thoroughly. As a developer myself, it's not that I don't trust engineers that work on my project. But rather because I'm an engineer, I know that sometimes you just happen to forget to build and run your tests. I've learned that the best way to get developers to conform to these standards is to simply block pull requests if the right thing isn't in there.

Effectively, we need to automate this accessibility testing and then block the pulled requests if they fail. If your work cycle looks something like this, where you make some code changes, you push them to some server and you create a request to merge changes into your main branch and then you merge them. But this would mean that we would have to insert one more step, to simply check if the component you've added or the code you've added is accessible. What that looks like is quite similar to the manual portion is that we would need to start up a server, launch a browser, go to localhost. I have no idea why that's a chicken in there. We'd have to wait for it to load, and then once it's loaded, we then have to run our Pelly tool.

To do this, because now this needs to happen in the command line, I found Pelly CI, which is an accessibility test runner that's built with Pelly under the hood. But this one is particularly focused on running in continuous integration environment. This is particularly important because we don't just want to get results of what's broken. We also want to break the build itself so that we don't allow it to be merged. Putting this into action, I chose to go with GitHub Actions, because that is probably one of the most commonly used CI-CD pipelines. GitHub Action automates tasks in the development workflow, which means that we can customize how we do this thing. Everything that I've just described, we have to put into our CI pipeline configuration.

But now if we talk about testing, a typical testing setup would look like this. You would need a test runner. Examples of this are Mocha and Jasmine. You need an assertion library which defines your testing logic. This says I expect A to come out as B under these conditions. Then because we're talking about things that actually need to be rendered, we also need a browser. Now, one of my favorite tools because I use it on all sorts of projects is Jest. Jest sits right across those two. It's both a test runner and an assertion library. They describe it as being a delightful JavaScript testing framework that focuses on simplicity. What that looks like is that if you have a function called sum, you would simply write a test that says, we expect the sum of these things to be two or to be this result.

7. Testing and Configuration

Short description:

If a test fails, it can be used to block the build. Gistx helps replace functions with component testing. It provides insights into violated rules and offers configuration options. Developers need guidance and coaching during development.

If it fails, as you see here, I said I expect the sum of one and two to be six, which we know it's incorrect. This is what the error would look like and it tells you that one of your test fails, which means that you can block the build using this.

I then realize that we can actually use gist with something called gistx. X is another core rules engine that helps you figure out accessibility issues. Once we start using it like this, we're able to replace the function with a render of a simple component that we want to test because we cannot always be testing entire pages. Sometimes we want to test a small component of what it is that we are building.

We see that once we pass it in there and we ask the gistx runner to test it for us, it's able to tell us which rules we've violated. But if you take a look at this one, it says that all page content should be contained by landmarks. This is great and true for full pages, but not necessarily always the case in smaller components that'll always exist in bigger pages. Which means that we need a way to configure these things, which is really, really great. You'll see that it also tells you what to fix and it gives you a link to Deque University's guidelines as to how this actually works. To configure it, it's nice and simple. We can enable the different rules for isolated components or for our different test cases because we know that we can never just have one solution for everyone. But again, this is a guideline, or this is a guard that we put when developers are building. What we really want to do is get even closer to the developer and help guide them and coach them while they're actually working.

8. X-Accessibility Ginter and Husky

Short description:

The X-Accessibility Ginter provides tooltips on why issues are marked and how to fix them. Husky is a GitHub tool that runs tests on commit to ensure accessibility. It can even block commits until accessibility issues are resolved.

For this, there's the X-Accessibility Ginter. It provides tooltips on why issues are being marked, why they are considered to be issues and even how to go and fix them. To go one step further at annoying my team, I also came across Husky, which is a beautiful GitHub that allows you to run these tests on commit. Not only are we stopping developers from merging, but if I really want to annoy the team, I'll configure this to stop their commits so that they don't even commit anything until they make sure that it's accessible. What it looks like is that once you try to commit, it will tell you that there's a problem here and this is why things are failing.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

DevOps.js Conf 2024DevOps.js Conf 2024
25 min
Atomic Deployment for JS Hipsters
Deploying an app is all but an easy process. You will encounter a lot of glitches and pain points to solve to have it working properly. The worst is: that now that you can deploy your app in production, how can't you also deploy all branches in the project to get access to live previews? And be able to do a fast-revert on-demand?Fortunately, the classic DevOps toolkit has all you need to achieve it without compromising your mental health. By expertly mixing Git, Unix tools, and API calls, and orchestrating all of them with JavaScript, you'll master the secret of safe atomic deployments.No more need to rely on commercial services: become the perfect tool master and netlifize your app right at home!
TestJS Summit 2021TestJS Summit 2021
36 min
Effective Performance Testing to your Server with Autocannon
Performance testing expertise that is developed for a long time. In order to measure your server performance you need a tool that can efficiently simulate a lot of abilities and give you good measurements according your analysing criteria.Autocannon NPM library gave me exactly that - that library is super easy to install and has a very simple API to work with. Within a really short amount of time you can start do performance testing to your application and get good measurements in development environment and in your performance labs, and generate complicated testing scenarios.In this talk I will introduce Autocannon, explain how to efficiently analyse your server performance with it, and show how it helped me to understand complicated performance issues in my Node.js servers. At the end of this lecture, developers will be able to have the ability to integrate a fast and easy tool in order to measure your server performance.
TestJS Summit 2021TestJS Summit 2021
30 min
Configuring Axe Accessibility Tests
Axe-core is a popular accessibility testing engine that is used Google, Microsoft, and hundreds of other companies to ensure that their websites are accessible. Axe-core can even integrate into many popular testing frameworks, tools, and IDEs. In this advanced session, we'll be learning how to configure axe and its integrations to fine tune how it runs and checks your pages and code for accessibility violations.
TestJS Summit 2022TestJS Summit 2022
21 min
Delightful Integration Tests With Testcontainers
Dockerized services are an excellent tool for creating repeatable, isolated environments ideal for integration tests. In this session, we'll look at the Testcontainers libraries which provide flexible and intuitive API for programmatically controlling lifecycle of your service dependencies in Docker containers. Running databases, Kafka, Elasticsearch, and even cloud technologies, straight from your test code ensures environment config is always up-to-date and consistent during local development and in CI pipelines.You’ll learn everything necessary to start adding powerful integration tests to your codebase without the headache of managing external service dependencies manually!

Workshops on related topic

React Summit 2023React Summit 2023
109 min
Web Accessibility for Ninjas: A Practical Approach for Creating Accessible Web Applications
In this hands-on workshop, we’ll equip you with the tools and techniques you need to create accessible web applications. We’ll explore the principles of inclusive design and learn how to test our websites using assistive technology to ensure that they work for everyone.
We’ll cover topics such as semantic markup, ARIA roles, accessible forms, and navigation, and then dive into coding exercises where you’ll get to apply what you’ve learned. We’ll use automated testing tools to validate our work and ensure that we meet accessibility standards.
By the end of this workshop, you’ll be equipped with the knowledge and skills to create accessible websites that work for everyone, and you’ll have hands-on experience using the latest techniques and tools for inclusive design and testing. Join us for this awesome coding workshop and become a ninja in web accessibility and inclusive design!
TestJS Summit 2021TestJS Summit 2021
85 min
Automated accessibility testing with jest-axe and Lighthouse CI
Do your automated tests include a11y checks? This workshop will cover how to get started with jest-axe to detect code-based accessibility violations, and Lighthouse CI to validate the accessibility of fully rendered pages. No amount of automated tests can replace manual accessibility testing, but these checks will make sure that your manual testers aren't doing more work than they need to.
React Summit 2022React Summit 2022
161 min
Web Accessibility in JavaScript Apps
Often we see JavaScript damaging the accessibility of a website. In this workshop, you’ll learn how to avoid common mistakes and how to use JS in your favor to actually enhance the accessibility of your web apps!
In this workshop we’ll explore multiple real-world examples with accessibility no-nos, and you'll learn how to make them work for people using a mouse or a keyboard. You’ll also learn how screen readers are used, and I'll show you that there's no reason to be afraid of using one!
Join me and let me show you how accessibility doesn't limit your solutions or skills. On the contrary, it will make them more inclusive!
By the end, you will:- Understand WCAG principles and how they're organized- Know common cases where JavaScript is essential to accessibility- Create inclusive links, buttons and toggleble elements- Use live regions for errors and loading states- Integrate accessibility into your team workflow right away- Realize that creating accessible websites isn’t as hard as it sounds ;)
TestJS Summit 2022TestJS Summit 2022
163 min
Automated Testing Using WebdriverIO
In this workshop, I cover not only what WebdriverIO can do, but also how you'll be using it day-to-day. I've built the exercises around real-world scenarios that demonstrate how you would actually set things up. It's not just "what to do," but specifically "how to get there." We'll cover the fundamentals of Automated UI testing so you can write maintainable, useful tests for your website and/or web app.
TestJS Summit 2021TestJS Summit 2021
111 min
JS Security Testing Automation for Developers on Every Build
As a developer, you need to deliver fast, and you simply don't have the time to constantly think about security. Still, if something goes wrong it's your job to fix it, but security testing blocks your automation, creates bottlenecks and just delays releases...but it doesn't have to...

NeuraLegion's developer-first Dynamic Application Security Testing (DAST) scanner enables developers to detect, prioritise and remediate security issues EARLY, on every commit, with NO false positives/alerts, without slowing you down.

Join this workshop to learn different ways developers can access Nexploit & start scanning without leaving the terminal!

We will be going through the set up end-to-end, whilst setting up a pipeline, running security tests and looking at the results.

Table of contents:
- What developer-first DAST (Dynamic Application Security Testing) actually is and how it works
- See where and how a modern, accurate dev-first DAST fits in the CI/CD
- Integrate NeuraLegion's Nexploit scanner with GitHub Actions
- Understand how modern applications, APIs and authentication mechanisms can be tested
- Fork a repo, set up a pipeline, run security tests and look at the results
GraphQL Galaxy 2021GraphQL Galaxy 2021
82 min
Security Testing Automation for Developers on Every Build
As a developer, you need to deliver fast, and you simply don't have the time to constantly think about security. Still, if something goes wrong it's your job to fix it, but security testing blocks your automation, creates bottlenecks and just delays releases, especially with graphQL...but it doesn't have to...

NeuraLegion's developer-first Dynamic Application Security Testing (DAST) scanner enables developers to detect, prioritise and remediate security issues EARLY, on every commit, with NO false positives / alerts, without slowing you down.

Join this workshop to learn different ways developers can access NeuraLegion's DAST scanner & start scanning without leaving the terminal!

We will be going through the set up end-to-end, whilst setting up a pipeline for a vulnerable GraphQL target, running security tests and looking at the results.

Table of contents:
- What developer-first DAST (Dynamic Application Security Testing) actually is and how it works
- See where and how a modern, accurate dev-first DAST fits in the CI/CD
- Integrate NeuraLegion's scanner with GitHub Actions
- Understand how modern applications, GraphQL and other APIs and authentication mechanisms can be tested
- Fork a repo, set up a pipeline, run security tests and look at the results