Tame the Component Multiverse

This is a talk about UI testing tooling problem that we critically underestimate and it's impact on both our applications our experience building them and I think the broader web in general. Now I want to talk about some tooling that I think is missing from our cool our current testing tool set. So we're gonna talk about some tools some strategies that I think that we should start employing as from developers and UI engineers and hopefully answer the question that some justfan has been asking since the second. I took this virtual stage. When will the CD take down this slide not quite yet. So I love the web and I love great software and I've been really disappointed as we've started to see the web take a bigger and bigger piece out of software development in general with the Advent of react native and electron. The web has started to move more and more places in software as a whole and it's starting to bring a pretty janky experience to everything across the board. Now selfishly. I want the web to be better because I use the web all the time. I love the web and I want to make sure that we have the test string strategies in place that ensure that we protect user experiences at every step of the way. So Now that we've got the intro out of the way, we can finally get rid of this slide. So who TF am I this is the point in the talk where I'm supposed to convince you that I have a lot of credibility and so I'm just gonna give you a general introduction to who I am and you can decide for yourself. If you find me trust where you're not so first of all, I'm Chan chantastic or Michael I go by anything. Whichever you prefer is fine with me. I used to you host a show called react podcast until unfortunately, I burned out it was just a ton of work. If you've ever done a podcast it's a lot of work but I got to speak during my time on react podcast with some of the just most brilliant developers and people I've ever met in my entire life. So if you're interested in that and you never heard of the show before there's a great catalog and if you want to hear from some of your favorite developers it's there for you. Now. I did kind of like in my burnout phase just fall in love with the game Destiny 2. So if you're Destiny player hit me up we can do some raids or Crucible together or something like that. Now was it all fun in games? I did get to work with the react course team on the react working group, which is a really awesome experience just bringing react 18 to life in the community and making sure that everyone knew how to transition their apps and take advantage of some of the new features. I also started one of my favorite online spaces for Creative curious and kind react to developers. That's a discord.gg slash lunch Dev. I'm a ton of people in there and I invite you to join if you like hanging out in discords. And then most relevant to this talk. I have about 12 years of experience inside the developer productivity Design Systems and front architecture space. That's really the the where I've cut my teeth in web development and where a lot of the experiences that I'll share today are going to come from I'm now working at chromatic and we'll talk about that service as we go through this talk and our goal. There is really just to improve the ux of the web. So if you've been in the reactant JavaScript space for a while, you've probably heard of the testing trophy. So let me put that on screen right now. This is the testing trophy as visualized by can't see Dodds. It's actually based on a tweet by guiromo Roush that says I think right tests not too much mostly integration. And that's kind of humorously enough is based on a summarization format that Michael Paulin used to summarize. This book The omnivore is dilemma, which is actually really good book and actually a really good kind of summarization format for a lot of things like testing. Anyway, that's where that comes from. This is the testing trophy. Now for this talk, I want to kick it on its side and talk about it as a Continuum and place some of our testing and testing strategies a top it so. When we look at this visualization, we have a handful things. We have static unit integration and end-to-end. These are all types of tests that we can write to support an application. Now in the realm of static typescript is dominating right now. So I'm just gonna put typescript over there and end to end it we hear a lot about Cyprus. They definitely have the Mind share in the middle. We have just or similar libraries kind of spanning the unit test and integration space now, I put Justin there because I think it's mostly well integrated with JS Dom and testing library with which are really popular tools in the react space specifically. So we have this is kind of where we're supposed to be writing. Most of our tests. Now. There is a little bit of complication here. We have two categories and three Tools in here. And anytime I see a distribution like this where it's not particularly clear what's happening. I wonder if we don't have two kids in a trench coat if there's not some kind of idea that is kind of a little bit fuzzy that we haven't quite pulled out yet. Now if I were to take a guess, I would say the concept that we haven't really mastered is visual testing. So that's what this is gonna be about visual testing. I'm talking about this slide and I'm gonna be a lot and I'm gonna be calling this the UI testing Continuum. So if I refer to that, this is kind of the the Continuum that I'm speaking of. Let's talk about the UI Multiverse. This is the problem that I see front end developers having to face in the visual visual integration testing space. Now. I also kind of kind of refer to this sometimes is the 10 ish dimensions of web UI or if I'm feeling particularly cheeky the 35,000 perfect States. Let's get into that. So anytime we design a web view we think about it in its ideal State this one perfect view, you know, kind of maybe we sketched it out on a piece of paper or we mocked it up, but there's one perfect ideal state for this view. Now, we all know that life is not perfect and we'll have to augment that with both error and loading State. We had a third dimension, which is the complications of loading and error states which you know, we might have dedicated skeleton views for Specific page and then we'd have different errors handled different ways because of 404 is significantly different from a 500 and so on and so forth. Now we have an additional Dimension even for a successful Pages which is break points. So we don't just make one single view typically for the web. We have a handful of breakpoints for a responsive website. It's pretty standard these days. Now, I typically run in the vicinity of six, but for the sake of this I'm gonna keep it small to four. Now that multiplies again when we start thinking about browsers and how our views actually interact with the browser. And right now we have you know, typically three or four engines that most applications have to Target and so I'll put four there. Now we take that and we multiply that again times you use your abilities. So this could be keyboard navigation or Mouse navigation or you know here having the site read via a screen reader. We also have to Vice capabilities. Now. This is a little bit different than user capabilities because a device can have touch or keyboard or mouse or can be spoken to and those are all ways that we can interact with the content now, I don't have any solutions for this part of it today. So I'm gonna not adding any slides to our expanding Universe of pages. So free pass on that one now eight most applications these days at least ones that kind of are Services we're gonna have some level of authorization as well. And I've just kind of duplicated this a handful of times to show different authorization types. But in addition to this you think about logic in your pages any page that has you know, like a Boolean the the changes the view, but what I haven't represented here is combinatorial logic where you have two props that kind of make a third or fourth or fifth possible state, so I know that you might be thinking like oh, wow, you just added a bunch of pages, but this is actually like a pretty minimal number when you consider how complicated components get and we'll talk about that a little bit more in a second. Now if you support multiple languages, then you really are gonna multiply this entire stack of perfect views times every language that you support. This is particularly important. If you support right to left where the page will actually flip effectively for your two for the proper language support. So pretty big pretty big numbers we're talking about here. Now if you work in this space of Design Systems or component libraries, you know that there's also this dimension of organization where you might have the whole world under your control, but you're Galaxy of components is actually under incredible pressure by the parts of your organization that adopted both intentionally and accidentally or unintentionally and how some of those teams or parts of your organization might even have the autonomy to choose the view layer that they use maybe they're not using a react I know in my case we're using rails and react but some of you might have parts of your application and backbone or Ember and our view even and building a lot of library of components that's comprehensive really involves supporting all of those libraries. so I can't really assume what challenges your particular design system and component Library are under how Ever I can tell you a little bit about mine and do the maths on the problems that I've been solving over the last handful of years and multiply that out. So these are the table Stakes so my application for every single view it would have six viewports three browser engines three device capabilities and four user abilities that brings us to 216. Perfect States. Well, there's a little bit more to it. We would have two authorization types at a minimum. Typically, it was more like four or five. We didn't support right to left because we're US based company, but we did support two Frameworks that brings us to a total of 864 perfect States, but it doesn't end there because we haven't even talked about Style. So we would support two themes two contrast modes and eat and all of those could be multiplied by the 10 applications, which would have a discrete theme for each application totally nutty but it actually brings us to a incomprehensible three 34,560 ideal States per component. Now this doesn't have handle. Even all of her authorization possibilities and definitely not any combinatorial logic from props absolutely mind boggling now. I want to talk about the the logic part of this for a second to just show you how much these UI states can grow exponentially. This was a blog post that demonstrated a button that was being built and all of the possible variants for not even additional contrast modes, but just two themes light and dark and all of the ways that a button could be represented in this application and they had 1134 variants. Absolutely nutty just for this one component. I think we have a really big problem and a lot of times we don't allow ourselves to feel the weight of this Challenge. And even right now you might be squirming because the first time they've actually thought of this all the way through is right now and a lot of times we get by by just not thinking about this by just focusing on our one task at hand and kind of letting QA figure out how to you know report back the errors to us or we just do Twitter driven development where we push it out knowing that there might be bad things and people will tell us where they're Now, I think that one of the biggest challenges we face is in test creation. So I want to talk a little bit about the suitability of testing strategies across this continuum. So first static is really great for interfaces. So languages are great for testing interfaces unit. Tests are great for testing into implementation of a unit of code. End-to-end tests are really great for flows sign up crud update billing Etc. And in this integration part we have coordination. Now ideally this is coordination of of components or code. They're separated from the full stack, which is what end-to-end tests. Now integration is primarily for I'm going to limit it to non-visual things and then visual is going to be that integration point where our code actually gets represented inside of a browser. About the suitability of these testing strategies by role. Now, I think in most cases we're going to see kind of engineering take a lot of the static unit and integration tests. It depends on your organization, obviously and on the other side QA is kind of largely dominated dominating the end to end side of things. But in this visual side, we have a lot of just kind of my and hopes that it looks good to us on our machine that it will look good to everyone else and that maybe those other two Fields will kind of close in the Gap a little bit for us. I think that if we want to have better test coverage in here, we need to have tools that work the way that our UI authoring does and make them quite a bit more to clear it. Now just on one side does have snapshot testing and Cyprus does have some component testing but the they have challenges in the way that they feel when authoring for visual tests. So how do we produce tests in a visual in this visual integration Gap and I think we do it automatically. Now I want to talk about a few things and I call this the declarative UI test declarative UI testing and so this will be a storybook chromatic and continuous integration. And I want to thank my coworker Varun for preparing these slides from a deck that he shares with our customers. So storybook when I think about storybook storybook is really sits in the gap between design and development and we have this idea of component-driven development where we actually really take advantage of components and the fact that they're isolated from the rest of the component environment. So we take a look we take a inspiration from tools like figma which have these ideas of sticker sheets for components and we bring that into the code side and we make this environment that is available to anyone with access to it. However, you want to share it that really visually represents the components and combines that with documentation and the ability to interact with them. So these are really living components. And we bring a lot of tools with us that we can share now in this front-end workspace environment. So we have view ports. We can codify that with the viewports that we support in our application. We can use tools like measure instead of having to open up an inspector and measure all this stuff. We just have that and make sure that all of our gaps are right the sizing is right between elements. We can outline things for debugging uis all inside of this front-end Workshop environment. Additionally we have accessibility tests built in this is typically something that you have to install either in your browser or through some kind of command line tool and we can bring that into our front end Workshop environment and make sure that everyone has the exact same information about the accessibility Audits and whether or not they're able to support the full spectrum of users in our applications. And for me, I know I do a lot of console log.whats or console.log watts, and we can have full a full breadth of ways to be able to test our components using controls or types Etc. We have automated automatically generated documentation inside of storybook that allows you to just by writing component have documentation that shows the interface of that component and the props available to you. And we have controls. So once those interfaces are identified. We now give you a playground. So anyone in different to their ability to actually code the component can Tinker around with some of the props and make sure that what they're trying to the button in this case that they're trying to achieve is possible with the interface available. Now there's this quote. I have the a chance to sit down with Ryan behan at Shopify and they focus a lot on the cross-functional power of their teams the ability for teams to focus on the the expertise that they have but have empathy for those other disciplines as well. And he says interactive storybook controls improve our teams cross-functional literacy and communication around react. I just love that. now chromatic is Is the piece of this where we can actually get some automated testing out of this storybook is really great at solving the organizational side of things but chromatic is where it becomes a testing tool. Now chromatic when you integrate with your CI pipeline is able to do a visual test against previously accepted visual tests. So when you upload your first one you kind of Define that as a Baseline and then every pull request after that will be compared to make sure that no unexpected changes have been introduced. Not only is that available for static views, but we can also do that for interactive things as well. So think about a modal here we can actually have a button that enacts a modal and then we can run our tests against the fact that that was actually clicked and opened pretty cool and we can do that across all of the browsers that we support as well and in all of the viewports and this is all integrated into your CI pipeline. So by sending a pull request now, you can run your army of robots on all of your your visual tests and make sure that nothing changed. Brad Frost who's just a Titan of the front end space put this really beautifully. He says testing his historically been really rough for us because it's been like picture in your mind's eye a modal and imagine clicking on a button that opens that modal. But by writing the story you get the documentation for the component tests and a playground all for free. Now I've been really excited to work with teams like Redwood as well to integrate even further storybook testing using testing Library MSW and Prisma into the framework and getting some really cool stuff out of it with things like redwoods generators, which would just generate all of these various states of Success login, you know error and loading States for you out of the box really cool. so we talked about storybook chromatic visual testing. Why should you care? Well, I believe that we are similarly cursed and that's because we love the web the web is awesome. We believe in everything everywhere all at once now and forever accessible to all and for all its faults no platform, but the open web really makes that promise. And if I'm being super honest, there's something fun about creating these worlds these universes that that I am fully in control of and when they push back I have to learn new strategies for kind of recontaining them again. And keep this universe this UI Multiverse that we've created under control. Unfortunately, it stops getting fun when things get out of control and we tend to burn out in kind of interesting ways. Unfortunately when we want to push new features, but every time we push something a hundred new bug reports come in or like 10 people complain about what we've done and it makes a sheepish about what we want to put out into the world. So we start saying no to new ideas features in the libraries because we're afraid that they'll break. That's when our products start to languish and our teams start to get irritated. And inevitably we become these gnarled villains that we swore. We'd never become when we were bright-eyed and ambitious and believed that everything was possible. But just kind of sit back and enjoy the fact that there's really nothing that we can do that will be successful anymore. But I want the web to be great. I want my apps to be great. I want your apps to be great and it's especially your apps if I'm using if I'm a customer of your apps. now I think that up to this point, we haven't had the front end tools that really represent the way that we work the way that we work in a visual space one. That's Complicated by multiple rendering engines and multiple libraries and there's a huge impedance mismatch between the the declarative way that we write code and the imperative way that we're expected to write tests. Now before components, we really never had the proper tools to handle the integration of the integration with our code and the browsers. And in a declared in doing and writing tests in a declared way that doesn't require the full stack like an end-to-end test does. So visual testing is a whole category of browser-based integration testing that we're just barely scratching the surface of and these are the tools that I I use and love and recommend but I really just hope that you start to explore this space of visual integration testing in your applications. And doing so in such a way that actually integrates with the tools that were already using in this integration space. Now again, the capsulation isolation of the component really has empowered us to do stuff that we've never done before and integrate with the browser in a way that doesn't require the full stack. I don't have said that a couple times but I think it's just such an important thing to remember the visual tests are integration test with the browser, but without the full stack. Now one of my favorite quotes is by Sandy Matt's a hero developer of mine and they say tester the wall at your back. Now. I want you to have the security that you will never break UI for any user ever again that every decision you codified for every view for every breakpoint and user type is defended contained and tamed so you can move confidently into the future. We have a huge task ahead but armed with the right tools and testing paradigms. I look forward to the web head. Tamer UI Multiverse, and let's improve the ux the web together. I'm chantastic. You can find me at Chan dot Dev at chantastic discord.gg slash lunchev. And if you're interested in storybook you check us out at storybook.js.org at storybook JS and discord.gg slash storybook. Thanks so much for listening. I hope to see you online.