Playwright Can Do This?

Rate this content
Bookmark

Guaranteeing that your application doesn't break while constantly shipping new features is tough. Obviously, with a continually growing app or site, you can't test everything manually all the time!

Test automation and monitoring are crucial to avoiding shipping broken apps and sites. But what functionality should you test? When should you run your tests? And aren't complex test suites super slow?

In this session, we'll get our hands on Playwright, the end-to-end testing framework, and learn how to automate headless browsers to ensure that you confidently ship new features.

23 min
03 Nov, 2022

AI Generated Video Summary

Playwright is a powerful tool for end-to-end testing, offering support for all major browsers and platforms. It provides features like parallelization, built-in waiting, and assertions. Playwright allows for running tests on multiple browsers with a single command and has functionality for generating tests and performing visual regression testing. It also enables the manipulation of the network layer and loading internals of web pages. Best practices include using short and idempotent scripts, splitting user account flows into separate tests, and cleaning up after each test case.

1. Introduction to Playwright and End-to-End Testing

Short description:

Hello, TestJS Summit. It's time for another Playwright session. Playwright can do this. I'm Stefan from Berlin, Germany, even though I'm in Greece right now. I do front-end development, JavaScript development, for quite a little bit. And I work for a company that is called checkly, and we do API and end-to-end monitoring in the cloud.

Hello, TestJS Summit. It's time for another Playwright session. Playwright can do this.

And before we get started, let me quickly introduce myself. I'm Stefan from Berlin, Germany, even though I'm in Greece right now. I do front-end development, JavaScript development, for quite a little bit. And I work for a company that is called checkly, and we do API and end-to-end monitoring in the cloud. So if you want to maybe control and run Playwright on a schedule, in the cloud, to make sure that all your products are up and running at all times, you can have a look at checkly. And I'm sure you will find that interesting.

And let me tell you that I started testing over 10 years ago, and these were the technologies, the technology choices that I started playing around with. So my first UI and end-to-end test were written in Selenium. And then at some point, Phantom JS showed up, this new fancy JavaScript-driven framework or library to control headless WebKit. And then later on, there was Casper JS. So we had a thing with ghosts at the time, which was a testing framework on top of Phantom JS. And let me tell you that from the beginning, from the first possibilities to control a browser to test that the stuff that I actually put online works, I was a big fan of this approach because the idea of end-to-end testing was always appealing to me. Because you can test your websites and applications really from end-to-end, starting at the browser and maybe ending at the database and testing and covering all the things in between. Let it be APIs, web servers, or whatever you're building.

2. Challenges with End-to-End Testing

Short description:

When I wrote my first end-to-end test, it was a terrible experience. The test suite was slow, flaky, and we didn't enjoy writing tests. Waiting for green lights and dealing with false positives made it even worse. We invested a lot of time but realized it's not worth it. Running tests on-demand is not enough. We want tests running all the time, on commits, on deploys.

When I wrote my first end-to-end test, though, it was a terrible experience. I spent, for example, sprint over sprint over sprint with my colleagues and my team to come up with a good end-to-end test coverage, only to create a test suite that was slow, we didn't enjoy writing tests, and was flaky. It was the absolute worst-case scenario to end up with a slow test suite so that you have to wait 30, 40, 50 minutes to get a green light to maybe deploy a typo fix, but then also get false positives results so that you re-run your end-to-end tests to make sure that well, maybe the tests just were not right, and then they paced later on, you probably have been there. This is just the worst-case scenario that when you cannot rely on your test suite, and it is also slow. So we were there, we invested a lot of time, and at some point we decided that it's not worth it, and we went for, hey, let's run the tests on-demand, right? And maybe you have been there, too, but this is the moment when you pretty much give up on end-to-end testing, because what you want to do is you want to have your tests running all the time, on commits, on deploys, you want them to be on your radar at all times. And when you run them on-demand, well, that is probably not often enough, so that you will end up with an outdated test suite very, very quickly, because you forgot about it, and it's kind of negating all the effort that you put into place. And this is exactly what happened to me.

3. Introduction to Playwright Testing

Short description:

Puppeteer and Cypress were the first steps in browser automation, but Playwright is a very decent solution for end-to-end testing. It is inclusive, supporting all major browsers and multiple platforms. Playwright Test is a full-fledged testing framework with convenient features like parallelization and built-in waiting and assertions. Let's have a look at Playwright and run some tests in VS Code.

Luckily, things got a lot better. Puppeteer was the first step for official browser automation, backed by Google for Chrome, and then Cypress a few years ago appeared with solid developer experience, and you might have guessed it, for a long time now, I'm playing around with Playwright, and I think it's just a very, very decent solution for end-to-end testing.

So, let me show you what Playwright is about. In case you haven't seen Debbie's talk, so first of all, Debbie from the Playwright team talked about Playwright just a few moments ago. Playwright is a very inclusive testing solution. So, it includes all major browsers, covering Chromium, WebKit, and Gecko Firefox. It runs on Mac, Windows, and Linux, and you can write tests primarily in JavaScript and TypeScript. If .NET, Python, and Java are your kind of thing, you can do that, too. So, from an inclusivity standpoint, Playwright is top-notch. Additionally, if VS Code is your editor of choice, the VS Code Playwright extension plays very nicely to create Playwright tests, debug them, generate them, run them. You can do all this end-to-end functionality and testing functionality right from your editor. If this is your jam, you can use VS Code, so that is very nice.

And it took me a while to realize that Playwright is actually more than a browser control. Playwright Test, these days, is a full-fledged testing framework that comes with the usual describe functionality, before each, after each hooks, fixtures, test configuration. All these goodies are included in Playwright itself. So when you want to really go all-in on end-to-end testing, Playwright is a valuable solution here. It's easy to paralyze. So remember my story of running tests and waiting for 30, 40 minutes? With Playwright, you can paralyze all your tests with a single CLI argument or put it in your configuration to speed up what happens and what is going on. And by far, my favorite is that Playwright is built for quick execution with features such as auto-waiting and web-first assertions, because UI testing is usually that, well, you wait for something, you do something with it and then you wait for the next thing to appear to do something with it then. And usually this means that I had to put arbitrary delays here and there or manually wait for elements to appear or disappear. Playwright has all of this baked in and this gets me very excited. Debbie briefly touched on this, but I want to show you how this works in this session too. So, shall we have a look? Let's have a look at Playwright and run some tests. So, we go to VS Code and what you see here is on the right side my terminal. And on the left side, we have test.js spec.js file, which is a standard test file. And what we will test is this lovely website here, which is on localhost 8080. And it's the test.js summit site and it's pretty much the same side, but now it includes this huge celebrate button, because I think more websites should have confetti in them. Other than that, when we look at the project configuration of the project setup here, you will find a standard playwright config here and other than that, there's a test folder including test.js spec.js and this little facebomb.js is just my notes here in case you're wondering that. We're not using the VSCode extension here, so to run this project and run the tests that are included in it, what you can do is npx playwright test and you see already that there are 28 discovered tests using five workers and we see already parallelization going on here, but we're going to have a look at what happens here right now. We see that there are seven past tests and 21 skipped and when we look at the test file, we see that there's one official or enable test and the other three are skipped, but why is everything multiplied by seven? So when we go to the playwright config, you will see that right now there are various and multiple browsers configured to be run when we run playwright tests.

4. Running Multiple Browsers

Short description:

You can run multiple browsers with a single command. It's cool to see all the windows popping up. We can disable most of the browsers to save time.

So there's Chromium, Firefox, WebCorp, Mobile Chrome, Mobile Safari, Microsoft Edge, Google Chrome. So this shows that you can run all of these browsers with a single command and to show you that this actually works, what we can do is we can pass the headed argument. And now we just see a bunch of windows popping up, emulating a specific browser and testing that all things are going well and are working properly. And I think this is already pretty, pretty cool to see all these windows popping up. But for now, I think we can just disable most of the browsers so that we don't have to wait for them going forward.

5. Generating Tests with Playwright

Short description:

Debbie showed how to generate tests using the Vscode extension or the command line. By running npx playwright codegen with a URL, the Playwright Inspector opens up and records all the actions. This provides a quick headstart for writing tests. Let's explore the tests further.

So what else do we have? Debbie showed in her talk that you can use the Vscode extension to generate tests. You can do the same thing from the command line pretty much too. So what I did here is I ran npx playwright codegen with a URL. And now what happened here is that the Playwright Inspector opened up. And now whatever we do in this little window here, let's click this year, let's do this and do this. You see that it's recording all the actions. I could now go here, copy and paste this, add some assertions and get a quick and a headstart using codegen if you want to do this. But let's have a look at the tests and start messing around with some tests.

6. Testing Page Title and Visual Regression

Short description:

So what we have here is a test that checks for the current page to have the correct title. We can run this test and it will fail because the title is incorrect. Playwright also comes with a debug mode for easy troubleshooting. Playwright has screenshot functionality for capturing the current state of the website. It also supports visual regression testing with one line of code.

So what we have here is first of all, a test that checks for the current page to have the correct title. And you see that there is a before each block that already goes to localhost. It checks the title, it locates the h1 element and then it does a string assertion that the headline has the correct test. So why not break this assertion here? Let's have a look at what happens.

So we can run this test and this will now fail because while tests, it will not have three Ss here and we'll retry a few times here. But what we now see is that we have a failed test. So how could we debug this? Even on the command line, playwright also comes with a debug mode. So what you see here is this browser session and again the player inspector that allows us to step through all the instructions of the actual tests here and to debug and see what is going on. And I think that's just a nice handy addition if you don't want to use the VS code extension.

So moving forward, what I want to show you is that first of all, playwright has the normal or the common screenshot functionality baked in. So when you're running a headless browser, very often you're wondering, Hey, what's the actual state of the website? So you can do the same thing. You can run a whitepage screenshot to take a screenshot of the current page and we now just generated a new PNG here, which includes the current status of this website. What is cool though, is that playwright has screenshot component, virtual regression testing baked in too. So what we also can do is what we have here is we have a locator for a headline. So let's do wait, expect. Now we can take the headline and we can say to have screenshot and let's give it the screenshot headline PNG. And when we, what happens here is that we are tying a, an assertion to a locator. We're going to screenshot this particular element that is matching this locator. And then for following runs, we're comparing the screenshots. So this is visual regression testing with one line of code. So when I now run this initially, it will fail because there is no snapshot, snapshot generated yet. But when we have a look at the result of this first failure, there is now a new directory that is testjess.spec.js.snapshots. And you see now here, this new snapshot that we just created. Going forward, when we now run the test again, it will always pass because Playwright will take a screenshot. We'll see if it's matching what we expect and moves on. So if we now adjust and mess around a little bit with this screenshot here, which is the baseline for comparison, and we run the test again, we will fail because we just messed with the comparison of the visual regression. And when we look into the test results folder here, now you see that this was the image that we expected with a lot of scratchy lines on it. This was the image that was generated in this current test run. And here we have the visual regression showing us, hey, something changed in your component that you have in your Playwright tests. And I'm still amazed that with a single line of tests instructions, you can have component-based visual regression testing.

7. Overwriting and Testing Confetti Functionality

Short description:

To overwrite what we just had, we can update snapshots. Let's test the confetti functionality. Playwright has built-in waiting functionality. We can leverage web-first assertions to check the presence of canvas elements. This streamlines end-to-end testing code. We can also mess with the network layer and loading internals of web pages.

To overwrite what we just had, what we can do is we can do update snapshots. And now we're back to the proper screenshot, which we will have here without scratch lines so that all our tests are passing. So this is good.

But let's move on and let's test that this very important confetti functionality is working. So what you see here is already that I have the button, that is the party confetti button. And I am locating all the canvas elements on this page. So when we look at this and we reload the page, you will see that there's actually quite a little bit of a delay for the button to appear. So what happens is that, first of all, we have to wait for this button to show up, then we have to click it, then we have to wait for the canvas to render all the confetti. And after the confetti is done, also, the canvas will be automatically removed. And this includes a lot of waiting.

If we would like to test this, but the nice thing about this is that playwright has all the waiting functionality built in. So what we can do is we can leverage the web-first assertions that come with Expect and we can say that, initially, we expect that there is no canvas element. Then we want to click the button and then we expect that there is one canvas element and that there is no canvas element anymore. And you see here that all of these instructions are asynchronous. We have to wait them. This looks like sequential code, even though there's a lot of waiting included here. So this button will only show up after a couple of seconds. Then the canvas element will immediately show up, but it will only disappear after a couple of seconds. And in PlayWrite, all of this is baked into the instruction itself. And I think this is just a wonderful way to streamline end-to-end testing code without putting waitforce and artificial delays everywhere. And you see here that we now have two past tests and we can break this by doing this, but I think that's just a beautiful and nice way. So we can move on.

And I want to show you something that I think is very, very cool too. So let's just fix the test again because it was broken now. So what you always can do in your test cases is you can always mess around with the network layer or with internals of the loading of a particular web page. So what you see here is that this test case get the page object in this test run. And you see here that we can listen for all requests and responses going in and out. So if we, for example, want to check the quality of a website and evaluate that there are no 404s or no 500s in API calls, what we can do is we can do a very quick status check. If there's a status response that is greater equal 400, we want to collect the URL and push it into an array that is not there yet. And let's create the array very quickly, const not found and do this.

8. Leveraging Native Browser APIs

Short description:

And we have to push this, here we go. And now we have to reload the page because it was already loaded. And then we add a quick assertion, expect not found length to be zero. This is all it takes to make sure that there are no 404 requests on your page. Our confetti test is slowing things down now a little bit, but it looks like we don't have any 404 resources or 500 in our little local house websites. You can also start blocking requests to speed up your end-to-end tests. Lastly, I want to show you how you can leverage native browser APIs to ensure your website is fast enough.

And we have to push this, here we go. And now we have to reload the page because it was already loaded. And then we add a quick assertion, expect not found length to be zero. And this is all it takes to make sure that there are no 404 requests on your page.

So we now run this test case again. Let's see what happens here. Our confetti test is slowing things down now a little bit, but it looks like we don't have any 404 resources or 500 in our little local house websites.

You can also do further stuff. You can start blocking requests. For example, if you want to speed up your end-to-end tests by blocking fonts or blocking tracking or blocking third party requests, because they are not necessarily needed in your end-to-end test suite at this moment in time, you can do that too with all the internals that PlayWrite provides you with the page on object.

And then lastly, I want to show you how you can leverage native browser APIs to make sure, for example, that your website is fast enough. So what you see here is that I use the page evaluate function, which is the way to implement your own little data aggregators inside of this particular website. So PlayWrite already provides a lot of functionality. But if there is something that you're missing and you want to hook into the context of the particular website, you can always use page evaluate and give it a function that runs in the context of this particular web test case in this website.

So what you see here is that I'm using page evaluate, and I'm instructing a new performance observer to figure out the metric largest contentful paint, which is then passed back to the testing scope. And what we now can do is we can do expect and I'm casting LCP to be a number because right now it's a string to be less than, I don't know, let's take 800 milliseconds here. Let's run this. Let's see if it's fast enough on my local machine here. Um, this looks cache. Let's see. Here we go. Cool. And these were just a few things that I wanted to show you how we can get started with playwright, running it on the command line. If the VS code extension is not your thing. So let's finish this up.

So really playwright comes with so much functionality, includes auto weighting, reference assertion screenshots, the test runner is nice. Retrying, tracing, VS code extension, inspector debugger coach, an image snapshots. There's really a bunch of functionality in there and the team is releasing a lot. So watch out. There's a lot of things, a good things coming.

9. Best Practices for Testing and Monitoring

Short description:

Short and idempotent scripts work better for testing and monitoring. Split your user account flow into different tests for parallelization. Have separate tests for login, update, and delete operations. Clean up after each test case to avoid cluttering the database.

But we at Checkli re-run playwright every few minutes for our customers, and I just want to share some common best practices. So first of all, what we've seen is that short and idempotent scripts usually work better for the testing and the monitoring use case. So instead of having and testing your entire user account flow, what you usually want to do is you want to split that in different tests to make that easily to parallelize. So what we often recommend all our customers is that they should have a create a login and update and a delete all for their account properties or resources so that all these kinds of things can run in parallel. And this then also means that if one fails and you really know what failed, what's going on, and you get a nice little message, Hey, this particular test case failed, and it's also very important that these kind of scripts are idempotent. What does that mean? I think it's a very fancy word. But this basically means that your test cases clean up after themselves so that you can run them repeatedly, which is very important for the monitoring use case. So whenever you create something, your test case should clean up after itself so that you're not cluttering a database, and these kind of test cases should run at all times.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Remix Conf Europe 2022Remix Conf Europe 2022
23 min
Scaling Up with Remix and Micro Frontends
Do you have a large product built by many teams? Are you struggling to release often? Did your frontend turn into a massive unmaintainable monolith? If, like me, you’ve answered yes to any of those questions, this talk is for you! I’ll show you exactly how you can build a micro frontend architecture with Remix to solve those challenges.
Remix Conf Europe 2022Remix Conf Europe 2022
37 min
Full Stack Components
Remix is a web framework that gives you the simple mental model of a Multi-Page App (MPA) but the power and capabilities of a Single-Page App (SPA). One of the big challenges of SPAs is network management resulting in a great deal of indirection and buggy code. This is especially noticeable in application state which Remix completely eliminates, but it's also an issue in individual components that communicate with a single-purpose backend endpoint (like a combobox search for example).
In this talk, Kent will demonstrate how Remix enables you to build complex UI components that are connected to a backend in the simplest and most powerful way you've ever seen. Leaving you time to chill with your family or whatever else you do for fun.
JSNation Live 2021JSNation Live 2021
29 min
Making JavaScript on WebAssembly Fast
JavaScript in the browser runs many times faster than it did two decades ago. And that happened because the browser vendors spent that time working on intensive performance optimizations in their JavaScript engines.Because of this optimization work, JavaScript is now running in many places besides the browser. But there are still some environments where the JS engines can’t apply those optimizations in the right way to make things fast.We’re working to solve this, beginning a whole new wave of JavaScript optimization work. We’re improving JavaScript performance for entirely different environments, where different rules apply. And this is possible because of WebAssembly. In this talk, I'll explain how this all works and what's coming next.
React Summit 2023React Summit 2023
24 min
Debugging JS
As developers, we spend much of our time debugging apps - often code we didn't even write. Sadly, few developers have ever been taught how to approach debugging - it's something most of us learn through painful experience.  The good news is you _can_ learn how to debug effectively, and there's several key techniques and tools you can use for debugging JS and React apps.
TestJS Summit 2022TestJS Summit 2022
27 min
Full-Circle Testing With Cypress
Cypress has taken the world by storm by brining an easy to use tool for end to end testing. It’s capabilities have proven to be be useful for creating stable tests for frontend applications. But end to end testing is just a small part of testing efforts. What about your API? What about your components? Well, in my talk I would like to show you how we can start with end-to-end tests, go deeper with component testing and then move up to testing our API, circ
TestJS Summit 2022TestJS Summit 2022
20 min
Testing Web Applications with Playwright
Testing is hard, testing takes time to learn and to write, and time is money. As developers we want to test. We know we should but we don't have time. So how can we get more developers to do testing? We can create better tools.Let me introduce you to Playwright - Reliable end-to-end cross browser testing for modern web apps, by Microsoft and fully open source. Playwright's codegen generates tests for you in JavaScript, TypeScript, Dot Net, Java or Python. Now you really have no excuses. It's time to play your tests wright.

Workshops on related topic

React Summit 2022React Summit 2022
117 min
Detox 101: How to write stable end-to-end tests for your React Native application
WorkshopFree
Compared to unit testing, end-to-end testing aims to interact with your application just like a real user. And as we all know it can be pretty challenging. Especially when we talk about Mobile applications.
Tests rely on many conditions and are considered to be slow and flaky. On the other hand - end-to-end tests can give the greatest confidence that your app is working. And if done right - can become an amazing tool for boosting developer velocity.
Detox is a gray-box end-to-end testing framework for mobile apps. Developed by Wix to solve the problem of slowness and flakiness and used by React Native itself as its E2E testing tool.
Join me on this workshop to learn how to make your mobile end-to-end tests with Detox rock.
Prerequisites- iOS/Android: MacOS Catalina or newer- Android only: Linux- Install before the workshop
React Day Berlin 2022React Day Berlin 2022
86 min
Using CodeMirror to Build a JavaScript Editor with Linting and AutoComplete
WorkshopFree
Using a library might seem easy at first glance, but how do you choose the right library? How do you upgrade an existing one? And how do you wade through the documentation to find what you want?
In this workshop, we’ll discuss all these finer points while going through a general example of building a code editor using CodeMirror in React. All while sharing some of the nuances our team learned about using this library and some problems we encountered.
TestJS Summit - January, 2021TestJS Summit - January, 2021
173 min
Testing Web Applications Using Cypress
WorkshopFree
This workshop will teach you the basics of writing useful end-to-end tests using Cypress Test Runner.
We will cover writing tests, covering every application feature, structuring tests, intercepting network requests, and setting up the backend data.
Anyone who knows JavaScript programming language and has NPM installed would be able to follow along.
Node Congress 2023Node Congress 2023
63 min
0 to Auth in an Hour Using NodeJS SDK
WorkshopFree
Passwordless authentication may seem complex, but it is simple to add it to any app using the right tool.
We will enhance a full-stack JS application (Node.JS backend + React frontend) to authenticate users with OAuth (social login) and One Time Passwords (email), including:- User authentication - Managing user interactions, returning session / refresh JWTs- Session management and validation - Storing the session for subsequent client requests, validating / refreshing sessions
At the end of the workshop, we will also touch on another approach to code authentication using frontend Descope Flows (drag-and-drop workflows), while keeping only session validation in the backend. With this, we will also show how easy it is to enable biometrics and other passwordless authentication methods.
Table of contents- A quick intro to core authentication concepts- Coding- Why passwordless matters
Prerequisites- IDE for your choice- Node 18 or higher
React Summit US 2023React Summit US 2023
96 min
Build a powerful DataGrid in few hours with Ag Grid
WorkshopFree
Does your React app need to efficiently display lots (and lots) of data in a grid? Do your users want to be able to search, sort, filter, and edit data? AG Grid is the best JavaScript grid in the world and is packed with features, highly performant, and extensible. In this workshop, you’ll learn how to get started with AG Grid, how we can enable sorting and filtering of data in the grid, cell rendering, and more. You will walk away from this free 3-hour workshop equipped with the knowledge for implementing AG Grid into your React application.
We all know that rolling our own grid solution is not easy, and let's be honest, is not something that we should be working on. We are focused on building a product and driving forward innovation. In this workshop, you'll see just how easy it is to get started with AG Grid.
Prerequisites: Basic React and JavaScript
Workshop level: Beginner
TestJS Summit 2023TestJS Summit 2023
89 min
Building out a meaningful test suite that's not all E2E
Workshop
We're all taught to follow the Testing Pyramid but the reality is that we build out the Testing Christmas Tree. In this workshop, David will talk you through how to break down projects and put the tests where they need to be. By the end of the workshop you will be able to update your projects so that anyone and everyone can start contributing and truly living up to "Quality is everyone job".
He will walk you through:- Component Testing- API Testing- Visual Regression Testing- A11Y testing
He will also talk you through how to get these all setup in your CI/CD pipeline so that you can get shorter and faster feedback loops.