Achieving A11y Automation Testing

Hello, everybody from around the world attending TestJS Summit 2021. I would love to speak to you today about some automation testing tools and strategies that everybody here hopefully can use starting today if they wish to around the topic of accessibility, specifically accessibility automation testing. This is a very exciting topic for me. I want to get one thing out of the way. If you see in my slides or I say out loud A11Y, which you can see on the screen right now, that simply means accessibility. A11Y, accessibility, they are synonymous. Brief explanation of why that is. There are 11 characters between the A and the Y in accessibility. If you've been working with internationalization in the past, you might be used to seeing I18N. It's the same thing. I18N for internationalization and A11Y for accessibility. But I digress. My name is Eva Gayde-Roten, and I am a lead full stack software engineer at Skills Engine where we build apps in Rails and in EmberJS. So there's plenty of JavaScript, which is where I put my focus, and that focus is mostly in the framework of EmberJS. A bit of this talk will be in EmberJS, which I imagine not too many people here are using, but I would like to share some of my background in that and some success story, and then we'll get into how to apply that to what it is that you are working on. So, as I just stated, we will, in this presentation, talk a little bit about EmberJS and specifically how it tackled accessibility. I believe it did a really good job, and I'll get into how and why. And then we will cover some tools and strategies for everyone, regardless of what you're using. You might be on React, maybe you're on Svelte or Vue, for example. Doesn't matter. We'll talk about those things. We'll also talk about how to automation test some of that, as well as generally taking ownership and being able to contribute, perhaps, back to some open source projects or the framework that you are working in. So last year, 2020, EmberConf. I spoke virtually at that conference about a slightly different success story around accessibility. That one was about how we were able to make a feature that we were shipping in our application even better by making accessibility a priority. We were able to have engineering, QA, design, and product all much happier with the thing that we delivered. We were able to deliver some automated tests around something that was click and draggy, and that's very difficult to write automation tests around. And because we added in accessibility, we were able to test that instead. That's a different topic for an update. But I have had multiple success stories, not just in Ember, but in accessibility as well. And I'll talk to you about some of that today. Ember has been embracing web standards for some time now. We like to say that we are very HTML first. We focus on W3C recommendations, MDN patterns, and HTML first allows us to inherently be very cognizant of accessibility. So let's get this out of the way. This is an important, I guess, let's talk about why accessibility is important, not just some stories that I've had. People with disabilities are the world's largest minority. There are some common things that go along with that. You might be able to argue that if you make apps that target people who have disabilities as well, you get more customers and more innovation in your product. And those kinds of things are very true. Those are legitimate, and those are valuable. But what I want to talk about is a little bit different. And I wanted to also say that people with disabilities is a broad term that speaks to many different things. It includes, but it's not limited to, visual, auditory, and motor disabilities as examples. But what I want to tell you all today is that accessibility doesn't need to be hard. Sure, doing it perfectly or even doing it extremely well, that's hard. Getting started, diving in, making some of it happen doesn't need to be hard. It can be automated. And furthermore, it can make your work that you do on a day-to-day even easier. So how did EmberJS embrace accessibility? I'd like to introduce two friends of mine. These are the mascots for Ember. On the left, we have Tomster. And on the right, we have Zoe. And they are wearing their accessibility shirts very proudly. We have some core support in EmberJS around accessibility. And this makes the biggest difference. I'd like to call out a wonderful developer by the name of Melanie Sumner. She is a senior engineer at LinkedIn. She has her hands in the pots of a lot of standards that are written for the web, as well as being a core team member, giving us first-class support for things that are important to her, such as accessibility. This means that in EmberJS, there is a deep respect for accessibility. One of the things that Melanie Sumner was able to put together was an accessibility working group, where we discussed some of the accessibility pitfalls that were currently existing. And we were able to get together in an open-source way, where we were logging our notes up on GitHub, and we were able to hop into Discord, schedule meetings, talk about things, go offline, be asynchronous, and all the great things that working open-source like will give you. It was really great. We were able to address some project issues, whether they were logged by the open-source community or by us, as we identified things that were just fundamentally lacking. Part of the thing was that we acknowledged that at the underlying framework level, every app that is built with Ember, if there are pitfalls, then all these apps could potentially have pitfalls. And so we worked to address those things. So we worked with issue logging and addressing those issues. We worked with writing up some RFPs, which is a fairly invested way of basically promising and explaining how you're going to solve something that's neither here nor there. Ultimately, we got to the point of being able to solve some of these issues, assigning out these tasks to various different people in the community. And we worked very heavily on guides and documentation. Every framework has a section of documents that will explain various different things, and each one I've noticed has at least something on accessibility. I'm very proud of what EmberJS has put together around it. There's also an additional site that I helped with writing some articles around, and it's about Ember component patterns, which is a way of saying patterns and anti-patterns, just like MDN does, the Mozilla Developer Network. And we're able to focus in on not just teaching how to write components that are HTML-first and W3C-compliant, but are accessible as well. Lot of great things there. Along with having first-class support, there is also an Ember accessibility-specific GitHub organization, which was a great place to house a lot of this work that we were doing, and a lot of the add-ons, which is what we call plugins or tools that can extend an Ember application. It makes a lot of the work that these different apps probably need to do around accessibility much easier. I also want to point out that a couple of these managed to make its way into the core Ember framework after much discussion within the community. That was great as well. So getting into the good bit. Code. We have two major tools that help specifically with accessibility. The first is in test, so writing some automated tests. We have an NPM package called Ember A11y testing, which will allow you to render out an entire page or a component, and then run an A11y or accessibility audit over that. This is great because you can write some green test, and then if an update happens in the future or somebody goes in and touches some code, it can fail your CI builds if those pages or those components are no longer accessible like they once were. And it can act as great checklists as well for the developer. There's also Ember template lint. This goes alongside things like ES lint, which will let you know when there are some warnings or errors in the code that you are typing. This one focuses on the templates, the HTML, and will let you know in your code editor as you are typing something like, hey, I have an image tag, and I forgot to write an alt attribute. When set up correctly, you can also have this fail in CI as well as it goes through and lints all of your files. So that's all great. Yay, applause for Ember. What we're here for is how to use these in other frameworks, how to take these tools and strategies and apply them to Vue, apply them to React and Svelte and anything else. So what's the first step? The first step is realizing that we need to prioritize accessibility. That means if you're in a 9-to-5, maybe you need to talk about it in your workplace with your team, with your product manager. Maybe it just means trying out some things. It also means in your communities, whether they are in person or online, open source projects you're really passionate about, or the frameworks themselves that maybe are lacking in certain areas and you can contribute to. That's why I'm here today, is to talk about it, talk about prioritizing accessibility, talking about why it's important, and spread the word a little bit. And everywhere we can do this is a step in the right direction. In the code again, there is a wonderful tool, it's an npm package that was published by Dex Systems called axe-core. This is made by a group that works on a lot of the standards around the web that are able to declare what are some warnings and violations for accessibility and then write some automation testing around it. Some of these tools can cover up to 50% of the accessibility rules that are out there, and that might not sound like a lot, but in some ways it really is, and it's a great start. Getting started with this kind of a tool is as simple as npm installing it, running axe, and then seeing what violations it has found. Now this is a core tool that is most often used to power other tools. And it's the foundation of so many other things, such as the browser extension. Browser extension is incredibly easy to get up and running with, no matter what browser you're running. This is also maintained by Dex Systems themselves, so it's got great support. And it can let you know exactly what it found on your fully rendered page or app, and what the priority is in fixing it, as well as maybe how to fix it. This is particularly great, not just for engineers, but QA, maybe design or product, to log these issues and whatever kind of ticketing system you might have, if that's relevant to you. Now framework integration is where it gets even more exciting. If you're using Svelte, congratulations, you already have accessibility built into that framework. It should be already outputting to your browser dev tools, I believe, some of these kinds of warnings. But if you're using React or Vue, it's as simple as installing an npm package for both of them, perhaps limiting down to just your development environment, or maybe a staging environment, if that is good for you. But it's just these simple lines of code. This is it. View use, view axe, and you're done. Now suddenly, you have something that is using axe core to send to your browser dev tools with very clear understanding of whether something is critical, serious, or moderate, a URL of how to figure out what to fix, et cetera. What's great about the framework integrations is that not only are they not too much work, but they do some of the heavy lifting of deduplicating some of these warnings and hooking into the underlying systems that say, hey, I have a component that just re-rendered, an update was fired, and they are able to intelligently recheck and see, hey, is there something else to be aware of, a new warning that I haven't already put into the developer tools? Now as far as linting goes, there is ESLint plugins for Vue.js accessibility and JSX11y. I'm sure there are others as well. And these are able to tell you directly in your code editor of choice, just like anything else in ESLint, hey, something you just typed or something in this file has a warning. And it's right there. You can see it and edit that file in order to make your adjustments and then see those warnings go away. It is so satisfying. And to be able to do this for accessibility, it's just an easy win. There's almost no reason not to do this. These are developer tools, which means that they are usually free to put into your application aside from a little bit of NPM install time because these don't even hit your users. But the effects of the things that you fix will. Let's talk about automation. I would argue that a lot of things I just talked about in some ways are kind of automating. It's not manual effort to go through and find some of these things. But if we want to talk about some true automation, maybe you're using Jest. Axe-core can plug in just by using Jest-axe. And then you can await axe, send us some HTML, and then expect it to have no violations. It's as easy as that. Once you have that in place, you can verify that within CI, within your testing environment, maybe within some component code, that that component or whatever it is you're working on is accessible. If you are using end-to-end testing frameworks, so things that are slightly separated from the framework itself that are rendering full URLs, perhaps on a staging environment or a QA environment, for example, perhaps WebDriver.io, there is a plugin for that. Axe-core, WebDriver.js, or if you are using Cypress-axe, there is a wonderful library for Cypress. And it's as easy as this. I'm not cutting out very much code on these slides. It's this simple. As far as communities go, as I mentioned earlier, a lot of frameworks have some great communities, some great documentation that are starting to come up. One of the most promising ones that I have seen so far is the Vue A11y community. They have a beautiful, very accessible website. It's great to start there with an accessible website, where they have so much potential for posts, recipes, tips, and tools. Their tools are the most built out, much like we had Ember add-ons that I was showing earlier. There are some Vue tools like Vue SkipTo or Vue Announcer that can help you handle some of these things that are a little bit more difficult to pull off without these add-ons kind of lending you a hand. This Vue A11y community is begging for some more contributions from the open source community. So I would say if you are into Vue or curious about Vue and accessibility, it's open source. Hop on there, take some of the pages that are placeholders right now that are just begging for some recipes or best practices around accessibility, write out some of those open support requests. The community will thank you, I'm sure. John at Sparkbox said, automated test as an enhancement, not a replacement for manual usability and accessibility testing. It's important to keep in mind after everything I just said and how excited I am about automated testing for accessibility to know that it's impossible to really get all the way there. Just as it's not reasonable to expect 100% test coverage on any application you're writing tests for. There's going to be some real user testing that you will have to do, some manual walking through the app and seeing how usable and accessible it really is using screen readers, using your keyboard to navigate through your applications. Perhaps the next time that you write a component, just see if you can tab through it. It's a good place to start. So today we talked about accessibility. We talked about why it's important and some success stories about how Ember was able to do it, how some other frameworks are starting to do it as well. We talked about making accessibility a priority in your workplace or your community. That might mean a framework that you have a lot of passion about. Or it might mean the team that you work with at your 9 to 5. We also talked about some AXE tools that are available for practically any environment. And if it's not available for yours, there's AXE Core. You might be the person to make that new NPM package that's powered by AXE Core to make something wonderful happen. Finally, I want to leave you with a personal request from me that is indirectly kind of related to this. And that request is to hire someone different than you if you have the means to do so. Your team will thank you. Your company will thank you. Future employees will thank you. Your users will end up thanking you. And I will. I want to say thank you for watching and joining me today. That is amazing. I do have to say that I was expecting the ones that were in the lead to maybe be in the lead. But I did not expect it to be this close across the board. I think my main reaction is that there are some places where it is legally required. People have to adhere by a standard, a particular quality of reaching some accessibility level. But a lot of the time, it's in the back of somebody's mind. I sure wish we could do accessibility. That's part of why I wanted to do this talk. So while you join me on stage, the camp that says it's unsure where to start is pulling ahead. So that's going to be my first question to you as we go into our Q&A session. Where do I start? I want to know more. Where do I start? Absolutely. I hope that this talk was able to provide a little bit of that. But where you start is talking with somebody on your team. Again, whether that is a 9 to 5 or some personal project that maybe it's just you or it's just you with your friends or an open source thing. It doesn't matter. Just talk with people. And make it important. Log some issues. Just say, like, hey, I don't think this was going to work for somebody who is using a screen reader. Just log those issues. Start talking about it. In terms of meaningfully, like, taking some action and, like, doing some code, basically, getting started with whatever your main tool is, your framework, followed by accessibility or acts, you're going to find these tools that I described and some other tools that I didn't get a chance to mention, like the ones for Angular, for example. That's good. We actually had someone that said that they want to know if the Ember tooling was a must or if there's any alternative to the Ember tooling. I can see a little Ember behind you, so I know you're an Ember fan. Yep. Guilty as charged. I have been doing Ember for far too many years now. And just along with that, I realized I have some success stories to share, some things I'm really proud of. But what I really want to be able to share is how these things can be applied in other places. So I hope that I was able to show some tools that are used for Svelte, Vue, React, and there are also ones out there for Angular. I just didn't want my slides to get too long. But there are tools out there for anything that you're using. That's good. Especially since accessibility, I think, is so important. There's some other questions in here. So I'm going to go through them. For example, would a linter be enough for basic accessibility testing or are most violations found after deployment in a more end-to-end style test? That's a great question. For accessibility problems, a linter will catch at least one of the cases, one of the types of problems that you will have. But there are probably two major cases. In the same way that will a JavaScript linter catch all of your JavaScript errors? No. But it will catch a certain case of them. You will still have undefined problems. Of course, I'm not talking about TypeScript. But you'll still run into other types of issues in the real world. Having a linter will allow me to realize, oh, I forgot to add a label around my input. I forgot to put accessible text on my image element. But it will not help somebody who's actually trying to flow through your entire form with the tab key and fill out everything just solely with a keyboard. That makes a lot of sense. That makes a lot of sense. There's a lot of questions. We won't be getting to all of them. I just want to remind everybody, Ava's going to be in her speaker's room afterwards. So you can ask the questions that I didn't get to in Spatial.chat. In the meantime, I think one of the questions says, how do I convince my manager that accessibility is important? If their argument is if users work in a restaurant, then accessibility is not an issue. And I think this is one of the most common sheets I've seen. Users work in some sort of whatever environment, be it a restaurant or you name it. It's not an issue. How do I convince my manager? There are several answers to this, but I'll try to get through them very quickly. You can use similar arguments that you have used for doing automation testing to begin with. I can argue very quickly that we don't have time for automation testing. Let's just keep shipping out code. But ultimately, it hurts your bottom line. And you can prove that mathematically if you really work at it and try to find those arguments. Also these kinds of things do apply to every different type of vertical, every different industry. For so long, we heard that mobile app development is not important. We don't have to worry about mobile. And then over time, we realized that was not the case. It's true that we need mobile for almost everything. Lastly, I want to say that if you can hire people that are different than who you are, like that was the last thing I said at the end of that talk, you're going to wind up with even more people with different perspectives, with different needs and desires and influences. And you might suddenly realize that your culture will change before your very eyes. I love that. I love that bit about culture. I think it's a really important piece of where this conversation starts, because it's one of those conversations that you'd hope it never comes up because you're doing the right thing. Thank you, Eva, for the Q&A session. I thought it was really good. I think I got through 20% of your questions. So whoever had questions that I didn't get to, please join Ava in the speaker's room on Spatial Chat. The link to join is in the same timeline for getting answers to all your questions.