Visual Regression with Puppeteer, Playwright and Cypress

Visual Regression tests components via screenshot matching. I'll show how you do that in three different libraries/frameworks. Additionally, I will use Storybook to extract the components from your SPA choice.


Transcript


Intro


Hello, and thank you very much for joining my talk. My name is Rainer Hahnekamp. I am from Austria, and I'm a trainer and consultant at Angular Architects. So this means I spend most of my time with Angular.

This talk is about visual regression, so this means that we will create automatically screenshots from our application, from our components, and we will match them against existing reference images. This is very similar to what we find with Jest snapshots, but the rest of the snapshots are based on strengths. Visual regression uses real images.

[00:58] And as I said before we are doing a lot of workshops and every time that I'm introducing this topic, there are two different types of developers. The first type says, "Well, that's great. I always wanted to see how this works. This is exactly what I needed." And it is very most likely that they are working on UI libraries, or that they have applications. A lot of time is invested into a very attractive design. The majority though says, "Well, I'm writing business applications. And most of my time I'm using form elements, I'm using Grids. And I don't create them by myself. I use already an the existing framework like material Bootstrap, or whatever. Why do I have to care about visual regression?"

If you're sharing this opinion, then just think what happens when you're entering a website and you see that the buttons are overloaded, overflown, or that a text is overflowing or something like that. Would you really trust this website? Would you really be ready to be for some of their services? Very likely not. Why? Well, because the first impression was already a very bad one. You don't trust them. And as everywhere in life, the first impression really counts. And as it is in applications, the visual part is always the first impression, so don't underestimate that regardless what type of application you have.

[02:36] And this is how we are going to do it. I will show you how to do visual regression by using three different libraries, frameworks. I will start with jest-Puppeteer, then Playwright, and then Cypress. Let's start.


Visual Regression


Storybook

[02:50] Okay, so here we are. This is our application and what we want to test is this holiday card here. And the first action is that we don't create a screenshot of this URL, because it also contains the site menu here on the left side, the header. And if something changes there and the test would throw an error, although the holiday card still looks nice. And because of that, we are using Storybook.

Storybook is a library that allows us to extract components out of our application, and to present or show them in an isolated way. And this is something that I have already done. So this is the Storybook instance. I can click through the different variations and that's what I'm going to use for visual regression. Let's take a quick look into our code, how this can be done.

[03:45] So Storybook requires a default configuration, in my case, this is Angular. I need to define all the dependencies. And then I just create the so-called stories, which are the different variations. And for that, I used here a self-written factory method.


Puppeteer

So we will start with Puppeteer. Puppeteer is a library that allows you to remotely control a chromium based browser, and this also allows you to make screenshots. And that's what we're going to combine here with Jest. You see here that I have already a Jest configuration that has a jest-puppeteer-preset. Most things are already configured by this preset, and in our package we have these two important dependencies. The one is of course, jest-puppeteer along with Puppeteer itself. And the second is image snapshot, which is able to do the matching of the two images. And that's everything that we need to do except to write the test. I have already prepared the test here. This is how it would look like, this is a parameterized test, because I am trying all the variations that I have. I am using the global page object to open a URL, navigate to my storybook instance. And once this is done, I am creating a screenshot from what I see, or what Jest sees. And then I just call the matcher to match in a snapshot. Let's run the test. And we see that since no screenshots are there yet, the Jest is creating new ones. And we find here this image snapshots directory if we open it, we see the screenshots.

[05:38] Now let's try to change something, and verify how it looks. So I'm in my CSS of my holiday card and I'm changing this from font weight to normal. So from bold to normal, we will see if our test will show us the difference now. We see that a lot of tests failed. Let's verify how we can see this. And we find in the image snapshots and further subdirectory, a diff output. And that shows us on the left side, the reference on the right side, the current one and in the middle, we see the difference.


Playwright

So the next thing is already a playwright, very quick here. What do we require for a playwright, or what is it?

[06:20] Playwright is very similar to puppeteer. It can be seen as a rewrite, then by Microsoft. It comes with its own testing library and it also supports visual regression out of the box. So we see here that dependency, there's just one package. We have a configuration. Here we can define multiple browsers. We also see here a web kit. So this is the test for Safari. And then we have the actual test. And as we see here, it looks more or less very similar to what we have seen before with Jest. Now let's run the test here as well.

In contrast to Jest or Puppeteer, the playwright counts cases where we don't have a reference image yet as a failure. So that's why we see here everything red. When we run it the next time, it'll only fail if there is a real difference. And of course the images are also there. Just need to open this directory, and you see here that it's there everything still.


Cypress

[07:21] And now to Cypress, Cypress is a very popular end-to-end framework. What do we require? Cypress of course, but also a Cypress plugin for snapshots. We need to add that snapshot in our support directory in the index, just import it. Also do the same in the plugin's directory itself. You see it here and you also want to update the cypress.json itself. There you need to add the configuration. What's left? Well, running the test of course. How does it look like? We are opening the storybook instance. We are then saying, "Well, I have the document and I want to match the snapshot from it." Let's try it out.

So the test is now running. We see the screenshot, we see everything's fine. Let's check for the snapshot itself. It's part here of the image snapshots directory. No surprise, works very familiar and that's it.

So I hope that my talk was useful. Thank you very much of course, for watching. My source code is available as well. We'll be posted afterwards and I'm already looking forward to the Q&A. See you there. Bye.

Rainer Hahnekamp
9 min

Check out more articles and videos

Workshops on related topic