Accessible CI/CD

Hi everyone. Welcome to my talk and let's discuss accessibility in a different context here at CIDI, shall we?

Okay. But first and foremost, a bit about myself. My name is Maya Min. I'm a senior software engineer at Microsoft Industry AI. Our group is focusing on leveraging different LM technologies to develop AI-integrated solutions and applications for industry specific. I've been in the industry for more than 10 years, focusing on the web and the front end. And lately, I just released a book called Learning View with Aurelie, which this book will teach you and guide you how to develop web applications using view as a front end framework in the context of TypeScript. And if you're interested in learning a new awesome framework, check it out. I'm also a Cloudinary ambassador, Google developer expert, and also an organizer of the View.js Israel community. You can follow me on Maya Chavin in LinkedIn or Twitter or follow my blog posts at mayachavin.com. And that's enough for myself.

Let's go on to our first question. We know about accessibility. We know what accessibility is important, but why accessibility testing and why it has to put in the context of CIDI. Okay. First and foremost, try to click on this button. You can, right? Because this button, even though it looks like a real button, it has the text click here to continue. It's not a real button. It's just an image. And this is exactly what we talked about when we mentioned accessibility. The button is not accessible. No one can use it. It means we have an accessibility issue. So here comes the definition of web accessibility. So web accessibility or accessibility is how you see, how a user can hear, and how a user can perceive your application so that they can understand, or at least come up to some understanding how to use, navigate, or interact with your web applications. This is the full flow that's similar to user experience in a lot of sense. And when I say accessibility, I usually say it's user experience because it's including accessibility inside. And but based on this, it's very hard to decide what is the compliance level for accessibility. This includes a lot of things here, listening, viewing, or understanding, right? So for that, there's a group that WCAG come up with a very, it's a standard guideline of compliance for accessibility.

They've been working very hard and it's become an industrial standard for enterprise and public domain to follow and to reflect on. So the normal level for WCAG, acceptable level for WCAG, accessibility compliance for enterprise and public domain is the version 2.1 level EE and above. So there's 3 level AA and AAA. But there's a lot of things mentioned in their website, and you may feel a bit overwhelmed when you go to the website because it's including a lot of explanation. It's very detailed explanations. So in this talk, I will just summarize the most common compliance standard and check that we need to follow. Color contrast, navigation, focus, landmark, content layout, 200% for 400%. You may think that 200% and 400% is not that common. In fact, it's very common because it included how the users zoom in and zoom out your website on a small device machine, like a small screen laptop. And 200% and 400% can define the maximum level the user can zoom in for your applications and make sure that your content stays consistently and usable even when users zoom in in that such case. Understandable content, form handling, media, typography, external internal link behavior. If you have an external link, which means the user will be redirected to a new page, open a new tab, you need to somehow inform the user that this is going to be navigating to a new website and the user won't be taken as surprised. Tab order and many more. Yes, many more. These are just an essential set of things to follow. And there's other things depending on how complex your application would be.

So that comes to the next question. Can we automate accessibility testing? Well, let's take a look at our components in UI applications. I like to use my favorite design pattern, atomic design, where we break components into smaller and smaller components to the lowest level. So we have atoms, which is the smallest isolated piece of core or component that is stand-alone and can be used for building a much more complex level of components like molecules. Molecules, built on top of atoms, can be used to build more complex organisms. So atoms, thinking like a button, an icon. Molecules, thinking like a form. And organisms, thinking like a gallery where you have a list of items and each item has another component. And then you also have field search, and pagination, and so on. And from this organism, you can build more and more complex, a full page. Page is less likely a component, but it's more like a stand-alone that you can do performance testing and end-to-end testing on this level.

So for accessibility compliance and accessibility testing for components, we also break it into according to different levels of a component. So for atoms, we don't really see the needs of testing everything. So for atoms, we only test color, focusable, semantics, HTML usage, informatic content, and typography. All the things that can be test isolated. It doesn't need to interact with other components. It can be interacted with by itself. So we check at this level only such things. When we move to molecules, which already have the interaction between one item and another item, we start to care about navigation flow, layout landmark, and focus flow. And if it's table and form, we also need to check if the table and form are built correctly when you do a navigation between one cell to another cell. How does it look, and how does it feel at the user? And for organism, you kind of repeat what molecules check for accessibility was, but you also add another level called content layout. Content layout is similar to landmark usage where the content, how the content flows, whether it flows correctly, whether it's understandable or not. And for pages, we just repeat what we test in our organization because they are very much the same, just a bit more complex. So for all of this, accessibility testing for component levels are divided. We come up with two sets of testing for accessibility. The first is component test, and the second is flow or user preference test.

The component test including what areas are labeled, whether, sorry, attributes, whether we're using the correct area label and attribute, and information. Semantic usage, are we using the right semantic or are we using deep for something that's not supposed to be deep? Focusability, color contrast, this is more about text versus background. Typography, accessible label, whether we have eternity accessible label for an image like an art text and so on. Form labels, each for each input have to have a unique form, and it cannot be missed.

Do we have enough form to tell user what the input doing is? Do we have enough label to tell user what the input doing? Well, this is component test. The second one is flow and user preference test, where we do a bit more about user preference. Navigation flow using keyboard. Focus flow, when we do focus, you do tab navigation from one interactive element to another, what is the focus landing, where the focus flow that doesn't make sense to user. Content quality, screen reader support, where you need to hear what the screen reader is saying about your element on your page and see whether it also makes sense and include all the information that it needs to be included. Image versus text contrast on a banner, this is much harder to test compared to the text versus background, and error flow and landmark.

So based on this division, we now can understand more or less which one is more suitable for automation test and which one we cannot really automate, we have to do manual test. So for component test, we can see that most of them are just checking about label attributes, semantic usage, content, the very descriptive content, and check about the contrast of text and background, so on. For that, we can easily achieve with automated test, including unit test or integration end-to-end test. For flow and user experience, it's a bit more tricky. It's not that straightforward because it relies a lot on how the user and how human analyze your website and your flow. So for this, you have to do manual test, and manual tests incur in smoke test, post-production, and design evaluation before developer actually going on and develop a feature or component. This design evaluation is happening at the phase where designer design the whole flow and getting the feedback from developer before they hand over to developer. Okay, for automated testing, we have a very nice tool called xCore. It's an open-source JavaScript package that was developed by Deque, which is an organization that's dedicated for accessibility. And this xCore has been used in Google Chrome and Edge, adapted by Chrome and Edge in the analyzer for the browser and the web performance, such as Lighthouse. And it's proven to be very, very efficient. It allows you to config different types, set of rules for accessibility check. And it also allows you to config what level of WCG standard you want to test on your applications, depending on your needs. And it's always up to date so you shouldn't be afraid and will go out of date. Okay, and lastly, it has a very wide support for different frameworks and testing frameworks, such as React and ClearWire and Cypress. So you easily can create an end-to-end test integration with these extensions. So for xCore, it's an open-source JavaScript library for accessibility testing. You can use it also at the unit testing in your unit test suite. However, it's for convenience that you can use Chess, which is an open-source package, npm package, that already wraps around xCore and provides also the custom chess matcher. And you don't have to write your own matcher to fit xCore standard. And this project is pretty easy, pretty straightforward to use. So you include it in your chess unit testing system in your applications and install it with ChessDom to make ChessDom mimicking simulations for your test. And then you can use the render functions to render the component that you want to test, or the page you want to test, and pass the component to the x function and wait for it to assert whether it has any violations or not. For end-to-end tests, we have much more things that are wider than just support.

Because end-to-end accessibility tests are mainly meant for end-to-end testing, where we have to see the actual layout and we can do the testing on actual layout, similar to how users perceive your application. So for end-to-end tests, for xCore, it has a package that goes with Playwright, Playwright.js, which is my favorite end-to-end testing framework because it's very fast and free and it's open source project. And it can run in a lot of different languages, not just JavaScript.

For Playwright, it also has a dedicated section for accessibility testing, how to integrate xCore in accessibility testing. So to use xCore in Playwright, we will install xCore xPlaywright package and we'll just import the xBuilder, which is the class function that gives us the instance, include only wrap around the page, and then we can check, we can analyze, we can trigger the function, analyze on it. So first, we will go to, we will navigate our test to the page that the target page in this test is a homepage and then we will wait for the xBuilder to scan and analyze the page instance that we will just go to, and then we finally assert if there's any evaluation or not. Okay, so using xCore and Playwright, we can cover the real-world browser testing, including different devices, different browsers, and see if it actually works and how it looks. We can also extend it, not just see if there's any violation, we can also extend it whether a component is visible, an element is visible, an element is focusable, because it's mimicking user actual experience.

So when you can also see if an event handler, when you click on it, what will happen, the dialogs open as expected on the page level accessibility rule check and you can also do the third-party widget app validations and color contrast, full color contrast, and add fully automated with predefined set of rule. So that is xCore with Playwright, and this is the example of how the violation would look. It will be an array of violations where it will give you some detail, the impact level, and all the information about the elements that have the violation. Okay, and that is xCore with Playwright as end-to-end test and also as unit test.

What about adding accessibility as part of the CI workflow? So if you're using GitLab, there's a very nice feature for accessibility testing which is accessibility testing workflow. It's included in your GitLab workflow. What you need to do is just download the template and include it in your applications, the YARN template as the workflow, it's on workflow for accessibility, then include it into your main workflow YARN plate. In here, you can define an accessibility stage and we can also provide some variables including all the URL you want to test for your application on accessibility and include the template. The template is working and in the back it will be on top of PA11Y framework for end-to-end testing, and this framework is a very convenient framework that has a lot of different packages that allow you to create your own custom dashboard for accessibility where you can see the violation and different level compliance and it's open source of course. So that is about GitLab CI workflow.

We're talking a lot about accessibility testing, automated testing, manual testing, what we can use to put an end-to-end and unit test. So where do we plug this in the CI CD workflow if we don't use GitLab workflow? What is CI CD? So CI CD stands for continuous integration and continuous deployment. So this is an example of how the flow goes between code development to continuous integration and continuous deployment. So literally as you can see here in the code development, we have a single source control where all the code will stay there and then developer can work on top of it with branches and merge back to the source. It can be GitHub, it can be GitLab, it can be Azure DevOps repo which they build on top of Git repository and the four levels of CI CD integration, the source, the build, the test, up the deploy.

What in the source we're talking about is the course version control. We also have some pipeline run on commit up here to make sure that nothing break or uses unit test whoops before we merge it back to source so we can control the quality of the source code. We also have coding standard operations such as LinkedIn, such as checklist, such as code review. All of this will make sure that our code is at least at a certain standard before we move to build. So build, we combine the actual source code into the actual consumable package such as NuGet package, such as ESC, such as JAR and a OS minify file so that we can deploy to the cloud for a web application. And in this build version, we're going to run a set of unit tests on top of the ready source code so we know that the source is stable enough to build. And then after we build, we will deploy these to a testing environment and then on that, we can perform more complex tests such as end-to-end test, integration test, performance and security test.

And finally, if things go well, we deploy to productions and the part of deployment have to be fully automated. And post deployment, we also have some tests called smoke test or where we actually have user feedback about product and we can do manual testing to see if in production it still look good and nothing break for user experience.

So where does accessibility testing fall into? So you can see here, it fall into all the highlight sections here. Code standard evaluations, unit test because we talk about unit tests for accessibility, end-to-end test, performance test and smoke test. Code standard evaluation is more about pre-code development and about co-review and collaboration between designer and developer. Unit test, end-to-end test, performance test is something we only talk about when we can plug in using Xcore or ChessX or Xcore with PlayLive for end-to-end test. And smoke test is manual testing on the user side or on the QA side for our product. That's how we integrated accessibility testing inside and we can build a CI-CD flow using Travis CI. Any Travis Circle CI checking Azure pipeline, any workflow, CI-CD workflow tool that can help you create a full flow for your application.