Visual Testing: Optimize Storybook and Win

Rate this content
Bookmark

One thing most people agree on is that UI's have multiple dimensions of complexity.Imagine you are at lunch with some of your team or family, and you get a Slack message that a portion of the site isn’t working.The Ad placement no longer loads on your money-generating news site that sees millions of monthly users.Or your login form loads infinitely after a user attempts to log in, and no one can access the application.The backend logic works and all API requests return 200 responses.It turns out we missed a loading state handler for the component, and the UI does not render correctly.How do we manage all of that?The conventional approach is to build the component on the application page where it is first used and manually test or use integration tests to verify UI.This approach makes it difficult to verify all the component states.Instead of all this overwhelm, you want a simple comparison of your UI and how it looks now after making changes.This is where Storybook + a visual regression testing tool come in!

9 min
07 Dec, 2023

AI Generated Video Summary

UI testing is hard, but Storybook and Chromatic simplify the process by allowing you to build and test your UI in isolation. Chromatic compares UI changes, detects even minor visual differences, and eliminates the need for manual testing. Loading images and simulating user workflows in Storybook ensure consistent testing. Interaction tests cover user workflows and can be used for complex components, resolving issues and creating test cases.

1. Introduction to Storybook and Chromatic

Short description:

UI testing is hard. There are two problems: broken UI and flaky integration tests. Storybook allows you to build your UI in isolation, simplifying the process. Chromatic creates automated tests of your UI. It ensures your UI isn't failing in production.

Hi, I'm Ruben, and I am the lead solutions engineer at Chromatic, and we're the maintainers of Storybook. So, I'm going to start with a story. This is you, and UI testing is hard. There are so many states and variants, responsiveness, loading states, it's all kind of confusing and it all burns down to the ground. Well, hopefully not, but that's how it ends up sometimes.

So, really, there are two problems. Let's say that this is your third week in a row that you've gotten a call and the UI is broken on the website and you need to go fix it, or even worse, the site is completely down. Now, that's not fun for anyone. And then, on the other side, when you're not getting called in on the weekend to fix UI issues, you're dealing with flaky integration tests, and that's not fun either. So, when do you have fun in your job? Doesn't sound like you're having fun at all.

Ideally, we move away from that complicated state and torch it and really make it simple. You just want what your UI looks like and compare it to what it looks like now. And storybook is the first part of solving that problem. Storybook allows you to build your UI in isolation, and it simplifies it because you don't have to worry about back-end logic and API calls, databases, none of that. You're just building your UI. And it works. So, it speeds up the process. And then, chromatic is the other side of that, where it creates automated tests of your UI. Now, what does that look like? I'm gonna show you an example.

So, here we have an application called Mealdrop, and it's an Uber Eats clone. Storybook allows you to create all types of components. You have your atomic components, so, for instance, this button, and storybook has controls which are essentially the props for your component and allows you to play around and prototype things all within the storybook environment in your browser. And you can also create page-level components. So, for instance, like the home page. Or a restaurant detail page. And this has everything in it, right? Your whole page structure. And the best part of it is that it's isolated, and you don't have to worry about inconsistent data or anything like that. Now, that solves the first problem. But it doesn't solve the second problem of how the heck... What do you do to make sure that your UI isn't failing in production? That's where chromatic comes in.

2. Chromatic UI Comparison

Short description:

Chromatic compares your UI by creating a baseline and taking snapshots of any code changes. It notifies you of any differences, eliminating the need for manual testing across browsers and devices. Chromatic can detect even minor visual changes that may go unnoticed by the human eye. Storybook and Chromatic work together to solve the challenge of taking static snapshots of dynamic UI, ensuring consistent testing without frequent adjustments.

Chromatic takes your statically built storybook, and it compares your UI. And how it does that is you first create a defined state or a baseline of what your UI looks like or what you expect it to look like. And then any time you change your code after that, it will take a new picture or a snapshot and compare it against the old snapshot. And any time there's a change, you get notified within chromatic, and then you no longer have to worry about manually testing, checking different browsers, different viewports, asking your co-worker for their Android phone and checking if it works on that. Yeah, I've done all of that. It's not fun.

So what's really neat here is chromatic can be as specific or as lax as you want it to be. By default, it's really strict. So seemingly, to the eye, nothing changed here. It looks exactly the same, right? If you focus in, before I even go to the section that's highlighted, chromatic knows that there's a slight difference in the image of the hamburger. And if you focus in on that hamburger, there are slight pixel changes. And so it will pick up that minor visual change that you wouldn't notice with your eyes. I mean, you might. My eyes aren't that good. So I would miss that.

Now this brings me to my second point. How do we solve for that in a... How do you take dynamic UI and take a static snapshot and make it work consistently so you're not having to fix all these tests every single time? And that's where Storybook comes in with chromatic. So you have these pages. If it loads. There we go. So the reason that that picture changes is because you have... So we're loading this picture from Unsplash, which is a network resource. And Unsplash can change that image at any time for any reason. The URL is exactly the same. So for all we know, nothing changed with this image. But really it did. And it could be more drastic than this. But luckily in this case it's not. It's just some minor pixels.

3. Loading Images and Simulating User Workflows

Short description:

Loading images within Storybook as a static asset ensures consistent snapshots. If you can't load images statically, you have a few options. You can mock the data or mock the network request using tools like mock service worker. Another option is to use play functions to wait for images to load. Interaction tests allow you to simulate user workflows in Storybook, saving time and effort.

And you might just approve it and move on. Well, that's not always the case. And so the best approach to this... There are actually several approaches. If you have the option to load images statically, then what we recommend is loading your images within Storybook as a static asset. That way your snapshots build by build are always consistent. Because you're always getting the same data and always getting the same assets.

Now sometimes you don't have that choice. You have maybe a source set from a CDN. And that's so you can get responsiveness and get different images loaded. And in those cases, we have to get a little bit more tactful about how we do that. So there are a couple options. You can completely mock the data that's coming in and pass mock data to your story. Or you can do a combination where you mock the network request with something like mock service worker. Which we learned about earlier today. And if you do that, you override the network request. So you're still getting a request. But you're saying, hey, when this request comes in, deliver this asset instead. And then the third option is to use the play functions built into Storybook to wait... You can create a utility function to wait for all of the images to load. And then put that in the play function. So you're telling the test to wait until your UI is in a state that you expect it to be.

Now, speaking of interaction tests... What's really neat about those is that you can simulate a user workflow in Storybook itself. So I'm gonna go to these templates. And oh, sorry. The user flows. And like... To get the success page to work in a non-simulated environment, you would have to take so many steps in order to get it working. But here I've defined 22 interaction steps with assertions and expect statements that have to pass in order for this test to pass in CI.

4. Interaction Tests and Complex Components

Short description:

Interaction tests cover the entire user workflow and ensure the correct rendering of the UI. They can also be used for more complex components. In one case, a dropdown modal couldn't be closed. By using Storybook and writing an interaction test, the issue was resolved. Now, there is a test case for both modal opening and closing.

So it covers your entire use case of going from front to back. And I'll even slow it down so you can see what happens. So if I rerun this interaction test, it starts from the very top and goes through the entire user workflow to make sure that this renders correctly each time. And you're testing that ending UI state that you expect your user to see when they run through this process.

Now, the last thing I want to touch on is that interaction tests aren't only for user workflows. You can also write interaction tests for more complex components. So in this case, I had a nav drop down that I was working on. And I was using Daisy UI and remix. And I could not get the modal to close. And I kept trying on my application and restarting and starting it again. And nothing was working. I was, like, oh, yeah, I work at Storybook. Why don't I have a Storybook for this? So I spun up a Storybook, wrote an interaction test until I got it passing and figured out the element that I needed to interact with in order to get it to close. And I fixed that. And so now I have not only a test case, a living test case that will test that my modal opens, but I can extend that in my close menu test. Which takes all of the code I've already written, adds a couple more steps to close the modal. And now I've got a test for both cases.

And that's it.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

TestJS Summit 2021TestJS Summit 2021
33 min
Network Requests with Cypress
Whether you're testing your UI or API, Cypress gives you all the tools needed to work with and manage network requests. This intermediate-level task demonstrates how to use the cy.request and cy.intercept commands to execute, spy on, and stub network requests while testing your application in the browser. Learn how the commands work as well as use cases for each, including best practices for testing and mocking your network requests.
TestJS Summit 2021TestJS Summit 2021
38 min
Testing Pyramid Makes Little Sense, What We Can Use Instead
Featured Video
The testing pyramid - the canonical shape of tests that defined what types of tests we need to write to make sure the app works - is ... obsolete. In this presentation, Roman Sandler and Gleb Bahmutov argue what the testing shape works better for today's web applications.
TestJS Summit 2022TestJS Summit 2022
27 min
Full-Circle Testing With Cypress
Cypress has taken the world by storm by brining an easy to use tool for end to end testing. It’s capabilities have proven to be be useful for creating stable tests for frontend applications. But end to end testing is just a small part of testing efforts. What about your API? What about your components? Well, in my talk I would like to show you how we can start with end-to-end tests, go deeper with component testing and then move up to testing our API, circ
TestJS Summit 2021TestJS Summit 2021
31 min
Test Effective Development
Developers want to sleep tight knowing they didn't break production. Companies want to be efficient in order to meet their customer needs faster and to gain competitive advantage sooner. We ALL want to be cost effective... or shall I say... TEST EFFECTIVE!But how do we do that?Are the "unit" and "integration" terminology serves us right?Or is it time for a change? When should we use either strategy to maximize our "test effectiveness"?In this talk I'll show you a brand new way to think about cost effective testing with new strategies and new testing terms!It’s time to go DEEPER!
TestJS Summit 2023TestJS Summit 2023
21 min
Everyone Can Easily Write Tests
Let’s take a look at how Playwright can help you get your end to end tests written with tools like Codegen that generate tests on user interaction. Let’s explore UI mode for a better developer experience and then go over some tips to make sure you don’t have flakey tests. Then let’s talk about how to get your tests up and running on CI, debugging on CI and scaling using shards.

Workshops on related topic

React Summit 2023React Summit 2023
151 min
Designing Effective Tests With React Testing Library
Featured Workshop
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents- The different kinds of React application tests, and where component tests fit in- A mental model for thinking about the inputs and outputs of the components you test- Options for selecting DOM elements to verify and interact with them- The value of mocks and why they shouldn’t be avoided- The challenges with asynchrony in RTL tests and how to handle them
Prerequisites- Familiarity with building applications with React- Basic experience writing automated tests with Jest or another unit testing framework- You do not need any experience with React Testing Library- Machine setup: Node LTS, Yarn
TestJS Summit 2022TestJS Summit 2022
146 min
How to Start With Cypress
Featured WorkshopFree
The web has evolved. Finally, testing has also. Cypress is a modern testing tool that answers the testing needs of modern web applications. It has been gaining a lot of traction in the last couple of years, gaining worldwide popularity. If you have been waiting to learn Cypress, wait no more! Filip Hric will guide you through the first steps on how to start using Cypress and set up a project on your own. The good news is, learning Cypress is incredibly easy. You'll write your first test in no time, and then you'll discover how to write a full end-to-end test for a modern web application. You'll learn the core concepts like retry-ability. Discover how to work and interact with your application and learn how to combine API and UI tests. Throughout this whole workshop, we will write code and do practical exercises. You will leave with a hands-on experience that you can translate to your own project.
React Summit 2022React Summit 2022
117 min
Detox 101: How to write stable end-to-end tests for your React Native application
WorkshopFree
Compared to unit testing, end-to-end testing aims to interact with your application just like a real user. And as we all know it can be pretty challenging. Especially when we talk about Mobile applications.
Tests rely on many conditions and are considered to be slow and flaky. On the other hand - end-to-end tests can give the greatest confidence that your app is working. And if done right - can become an amazing tool for boosting developer velocity.
Detox is a gray-box end-to-end testing framework for mobile apps. Developed by Wix to solve the problem of slowness and flakiness and used by React Native itself as its E2E testing tool.
Join me on this workshop to learn how to make your mobile end-to-end tests with Detox rock.
Prerequisites- iOS/Android: MacOS Catalina or newer- Android only: Linux- Install before the workshop
TestJS Summit 2023TestJS Summit 2023
48 min
API Testing with Postman Workshop
WorkshopFree
In the ever-evolving landscape of software development, ensuring the reliability and functionality of APIs has become paramount. "API Testing with Postman" is a comprehensive workshop designed to equip participants with the knowledge and skills needed to excel in API testing using Postman, a powerful tool widely adopted by professionals in the field. This workshop delves into the fundamentals of API testing, progresses to advanced testing techniques, and explores automation, performance testing, and multi-protocol support, providing attendees with a holistic understanding of API testing with Postman.
1. Welcome to Postman- Explaining the Postman User Interface (UI)2. Workspace and Collections Collaboration- Understanding Workspaces and their role in collaboration- Exploring the concept of Collections for organizing and executing API requests3. Introduction to API Testing- Covering the basics of API testing and its significance4. Variable Management- Managing environment, global, and collection variables- Utilizing scripting snippets for dynamic data5. Building Testing Workflows- Creating effective testing workflows for comprehensive testing- Utilizing the Collection Runner for test execution- Introduction to Postbot for automated testing6. Advanced Testing- Contract Testing for ensuring API contracts- Using Mock Servers for effective testing- Maximizing productivity with Collection/Workspace templates- Integration Testing and Regression Testing strategies7. Automation with Postman- Leveraging the Postman CLI for automation- Scheduled Runs for regular testing- Integrating Postman into CI/CD pipelines8. Performance Testing- Demonstrating performance testing capabilities (showing the desktop client)- Synchronizing tests with VS Code for streamlined development9. Exploring Advanced Features - Working with Multiple Protocols: GraphQL, gRPC, and more
Join us for this workshop to unlock the full potential of Postman for API testing, streamline your testing processes, and enhance the quality and reliability of your software. Whether you're a beginner or an experienced tester, this workshop will equip you with the skills needed to excel in API testing with Postman.
TestJS Summit - January, 2021TestJS Summit - January, 2021
173 min
Testing Web Applications Using Cypress
WorkshopFree
This workshop will teach you the basics of writing useful end-to-end tests using Cypress Test Runner.
We will cover writing tests, covering every application feature, structuring tests, intercepting network requests, and setting up the backend data.
Anyone who knows JavaScript programming language and has NPM installed would be able to follow along.
TestJS Summit 2023TestJS Summit 2023
148 min
Best Practices for Writing and Debugging Cypress Tests
Workshop
You probably know the story. You’ve created a couple of tests, and since you are using Cypress, you’ve done this pretty quickly. Seems like nothing is stopping you, but then – failed test. It wasn’t the app, wasn’t an error, the test was… flaky? Well yes. Test design is important no matter what tool you will use, Cypress included. The good news is that Cypress has a couple of tools behind its belt that can help you out. Join me on my workshop, where I’ll guide you away from the valley of anti-patterns into the fields of evergreen, stable tests. We’ll talk about common mistakes when writing your test as well as debug and unveil underlying problems. All with the goal of avoiding flakiness, and designing stable test.