Playwright Can Do This?

Hello, test your summit. It's time for another playwright session. Playwright can do this. And before we get started, let me quickly introduce myself. I'm Stefan from Berlin, Germany. Even though I'm in Greece right now. And I do front end development, JavaScript development for quite a little bit. And I work for a company that is called Checkli. And we do API and end-to-end monitoring in the cloud. So if you want to maybe control and run playwright on a schedule in the cloud to make sure that all your products are up and running at all times, you can have a look at Checkli. And I'm sure you will find that interesting. And let me tell you that I started testing, I don't know, over 10 years ago. And these were the technology choices that I started playing around with. So my first UI and end-to-end tests were written in Selenium. And then at some point, PhantomJS showed up. This new fancy JavaScript driven framework or library to control headless web kit. And then later on there was CasperJS. So we had a thing with ghosts at a time, which was a testing framework on top of PhantomJS. And let me tell you that from the beginning, from the first possibilities to control a browser, to test that the stuff that I actually put online works, I was a big fan of this approach. Because the idea of end-to-end testing was always appealing to me. Because you can test your websites and applications really from end-to-end, starting at the browser and maybe ending at the database and testing and covering all the things in between. Let it be APIs, web servers, or whatever you're building. When I wrote my first end-to-end test, though, it was a terrible experience. I spent, for example, sprint over sprint over sprint with my colleagues and my team to come up with a good end-to-end test coverage only to create a test suite that was slow, we didn't enjoy writing tests, and was flaky. And it was the absolute worst case scenario to end up with a slow test suite so that you have to wait 30, 40, 50 minutes to get a green light to maybe deploy a typo fix. But then also get false positives results so that you rerun your end-to-end tests to make sure that, well, maybe the tests just were not right and then they paced later on. You probably have been there. This is just the worst case scenario that when you cannot rely on your test suite and it is also slow. So we were there, we invested a lot of time, and at some point we decided that it's not worth it. And we went for, hey, let's run the tests on demand. And maybe you have been there too, but this is the moment when you pretty much give up on end-to-end testing, because what you want to do is you want to have your tests running all the time on commits, on deploys. You want them to be on your radar at all times. And when you run them on demand, well, that is probably not often enough so that you will end up with an outdated test suite very, very quickly because you forgot about it. And it's kind of negating all the effort that you put into place. And this is exactly what happened to me. Luckily, things got a lot better. Puppeteer was kind of the first step for official browser automation, baked by Google for Chrome. And then Cypress a few years ago appeared with solid developer experience. And you might've guessed it, well, over for a long time now, I'm playing around with Playwright. And I think it's just a very, very decent solution for end-to-end testing. So let me show you what Playwright is about. In case you haven't seen Debbie's talk. So first of all, Debbie from the Playwright team talked about Playwright just a few moments ago. Playwright is a very inclusive testing solution. So it includes all major browsers covering Chromium, WebKit, and Gecko Firefox. It runs on Mac, Windows, and Linux. And you can write tests primarily in JavaScript and TypeScript. But if.NET, Python, and Java are your kind of thing, you can do that too. So from an inclusivity standpoint, Playwright is top notch. Additionally, if VS Code is your editor of choice, the VS Code Playwright extension plays very nice leads to create player tests, debug them, generate them, run them. You can do all these end-to-end functionality and testing functionality right from your editor. So if this is your jam, you can use VS Code. So that is very nice. And it took me a while to realize that Playwright is actually more than a browser control. Playwright Test these days is a full-fledged testing framework that comes with the usual describe functionality before each, after each hooks, fixtures, test configuration. All these goodies are included in Playwright itself. So when you want to really go all in on end-to-end testing, Playwright is a valuable solution here. It's easy to paralyze. So remember my story of running tests and waiting for 30, 40 minutes. With Playwright, you can paralyze all your tests with a single CLI argument or put it in your configuration to speed up what happens and what is going on. And by far, my favorite is that Playwright is built for quick execution with features such as auto-waiting and web-first assertions. Because UI testing is usually that, well, you wait for something, you do something with it, and then you wait for the next thing to appear to do something with it then. And usually this meant that I had to put arbitrary delays here and there or manually wait for elements to appear or disappear. Playwright has all of this baked in, and this gets me very, very excited. Debbie briefly touched on this, but I want to show you how this works in this session too. So shall we have a look? Let's have a look at Playwright and run some tests. So we'll go to VS Code, and what you see here is on the right side, my terminal. And on the left side, we have test.js spec.js file, which is a standard test file. And what we will test is this lovely website here, which is on localhost 8080, and it's the test.js summit site. And it's pretty much the same site, but now it includes this huge celebrate button because I think more websites should have confetti in them. Other than that, when we look at the project configuration of the project setup here, you will find a standard Playwright config here. And other than that, there's a test folder, including test.js spec.js. And this little facebomb.js is just my notes here in case you're wondering that. So we're not using the VS Code extension here. So to run this project and run the tests that are included in it, what you can do is npx playwright test. And you see already that there are 28 discovered tests using five workers. So we see already parallelization going on here, but we'll have a look at what happens here right now. So we see that there are seven past tests and 21 skipped. And when we look at the test file, we see that there's one official or enable tests and the other three are skipped. But why is everything multiplied by seven? So when we go to the Playwright config, you will see that right now, there are various and multiple browsers configured to be run when we run Playwright tests. So there's Chromium, Firefox, WebCorp, Mobile Chrome, Mobile Safari, Microsoft Edge, Google Chrome. So this shows that you can run all of these browsers with a single command. And to show you that this actually works, is what we can do is we can pass the headed argument. And now we just see a bunch of windows popping up, emulating a specific browser and testing that all things are going well and are working properly. And I think this is already pretty, pretty cool to see all these windows popping up. But for now, I think we can just, let's disable most of the browsers so that we don't have to wait for them going forward. So what else do we have? Debbie showed in her talk that you can use the VS Code extension to generate tests. You can do the same thing from the command line pretty much too. So what I did here is I ran npx playwright code gen with a URL. And now what happened here is that the playwright inspector opened up. And now whatever we do in this little window here, let's click this here, let's do this and do this. You see that it's recording all the actions. So that could now go here, copy and paste this, add some assertions and get a quick and a headstart using CodeGen if you want to do this. But let's have a look at the tests and start messing around with some tests. So what we have here is first of all, a test that checks for the current page to have the correct title. And you see that there is a before each block that already goes to local host. Then it checks the title, it locates the H1 element, and then it does a string assertion that the headline has the correct test. So why not break this assertion here? Let's have a look at what happens. So we can run this test and this will now fail because well, tests, it will not have three S's here and we'll retry a few times here. But what we now see is that we have a failed test. So how could we debug this? Even on the command line, Playwright also comes with a debug mode. So what you see here is this browser session. And again, the Playwright inspector that allows us to step through all the instructions of the actual tests here and to debug and see what is going on. And I think that's just a nice handy addition if you don't want to use the VS Code extension. So moving forward, what I'm gonna show you is that first of all, Playwright has the normal or the common screenshot functionality baked in. So when you're running a headless browser, very often you're wondering, hey, what's the actual state of the website? So you can do the same thing. You can run a weight page screenshot to take a screenshot of the current page. And we now just generated a new PNG here, which includes the current status of this website. What is cool though, is that Playwright has screenshot component, virtual regression testing baked in too. So what we also can do is, what we have here is we have a locator for a headline. So let's do wait, expect. Now we can take the headline and we can say to have screenshot, and let's give it the screenshot headline PNG. And what happens here is that we are tying an assertion to a locator, we're gonna screenshot this particular element that is matching this locator. And then for following runs, we're comparing the screenshots. So this is visual regression testing with one line of code. So when I now run this, initially it will fail because there is no snapshot generated yet. But when we have a look at the result of this first failure, there is now a new directory that is test.js.spec.js.snapshots. And you see now here, this new snapshot that we just created. Going forward, when we now run the test again, it will always pass because Playwright will take a screenshot, we'll see if it's matching what we expect and moves on. So if we now adjust it, mess around a little bit with this screenshot here, which is the baseline for our comparison, and we run the test again, we will fail because we just messed with the comparison of the visual regression. And when we look into the test results folder here now, you see that this was the image that we expected with a lot of scratchy lines on it. This was the image that was generated in this current test run. And here we have the visual regression showing us, hey, something changed in your component that you have in your Playwright test. And I'm still amazed that with a single line of test instructions, you can have component-based visual regression testing. To override what we just had, what we can do is we can do update snapshots. And now we're back to the proper screenshot, which we will have here without scratch lines so that all our tests are passing. So this is good. But let's move on and let's test that this very important confetti functionality is working. So what you see here is already that I have the button, that is the party confetti button, and I am locating all the canvas elements on this page. So when we look at this and we reload the page, you will see that there's actually quite a little bit of a delay for the button to appear. So what happens is that first of all, we have to wait for this button to show up, then we have to click it, then we have to wait for the canvas to render all the confetti. And after the confetti is done, also the canvas will be automatically removed. And this includes a lot of waiting if we would like to test this. But the nice thing about this is that Playwright has all the waiting functionality built in. So what we can do is we can leverage the web-first assertions that come with expect. And we can say that initially we expect that there is no canvas element. Then we want to click the button, and then we expect that there is one canvas element and that there is no canvas element more. And you see here that all of these instructions are asynchronous, we have to wait them. But this looks like sequential code, even though there's a lot of waiting included here. So this button will only show up after a couple of seconds. Then the canvas element will immediately show up, but it will only disappear after a couple of seconds. And in Playwright, all of this is baked into the instructions itself. And I think this is just a wonderful way to streamline end-to-end testing code without putting wait for us and artificial delays everywhere. And you see here that we now have two pass tests and we can break this by doing this. But I think that's just a beautiful and nice way. So we can move on and I want to show you something that I think is very, very cool too. So let's just fix the test again because it was broken now. So what you always can do in your test cases is you can always mess around with the network layer or with internals of the loading of a particular webpage. So what you see here is that this test case get the page object in this test run. And you see here that we can listen for all requests and responses going in and out. So if we, for example, want to check the quality of a website and evaluate that there are no four fours or no 500s in API calls, what we can do is we can do a very quick status check. And if there's a status response that is greater equal 400, we want to collect a URL and push it into an array that is not there yet. And let's create the array very quickly. const not found and do this. And we have to push this. Here we go. And now we have to reload the page because it was already loaded. And then we add a quick assertion, expect not found length to be zero. And this is all it takes to make sure that there are no four four requests on your page. So let me now run this test case again. Let's see what happens here. Cool, our confetti test is slowing things down now a little bit, but it looks like we don't have any four four resources or 500s in our little local house websites. You can also do further stuff. You can start blocking requests. For example, if you want to speed up your end-to-end tests by blocking fonts or blocking tracking or blocking third party requests because they are not necessarily needed in your end-to-end test suite at this moment in time, you can do that too with all the internals that Playwright provides you with the page on object. And then lastly, I want to show you how you can leverage native browser APIs to make sure for example, that your website is fast enough. So what you see here is that I use the page evaluate function, which is the way to implement your own little data aggregators inside of this particular website. So Playwright already provides a lot of functionality, but if there is something that you're missing and you want to hook into the context of the particular website, you can always use page evaluate and give it a function that runs in the context of this particular web test case in this website. So what you see here is that I'm using page evaluate and I'm instructing a new performance observer to figure out the metric largest contentful paint, which is then passed back to the testing scope. And what we now can do is we can do expect and I'm casting LCP to be a number because right now it's a string to be less than, I don't know, let's take 800 milliseconds here. Let's run this. Let's see if it's fast enough on my local machine here. This looks okay-ish. Let's see. Here we go. Cool. And these were just a few things that I wanted to show you how you can get started with Playwright, running it on the command line if the VS Code extension is not your thing. So let's finish this up. So really Playwright comes with so much functionality, includes auto-weighting, reference assertion screenshots, the test runner is nice, retrying, tracing, VS Code extension, inspector debugger, code-jet image snapshots. There's really a bunch of functionality in there and the team is releasing a lot. So watch out, there's a lot of good things coming. But we at Shackley rerun Playwright every few minutes for our customers. And I just want to share some common best practices. So first of all, what we've seen is that short and idempotent scripts usually work better for the testing and the monitoring use case. So instead of having and testing your entire user account flow, what you usually want to do is you want to split that in different tests to make that easily to parallelize. So what we often recommend all our customers is that they should have a create, a login, an update and a delete, all for their account properties or resources so that all these kinds of things can run in parallel. And this then also means that if one fails, and you really know what failed, what's going on, and you get a nice little message, hey, this particular test case failed. And it's also very important that these kind of scripts are idempotent. What does that mean? I think it's a very fancy word, but this basically means that your test cases clean up after themselves so that you can run them repeatedly, which is very important for the monitoring use case. So whenever you create something, your test case should clean up after itself so that you're not cluttering your database. And these kind of test cases should run at all times. And what we do at Shackley is we pair our end-to-end tests with low-level API calls. So for example, if we create a resource with the UI end-to-end, we usually have teardown scripts that then clean up all the resources that were created during a specific test on the API level. And we run this very, very successfully. And then as a hot take here today, I would like to share that it would be great if we start treating our UIs like we do our APIs. So for example, in software, we always talk about API uptimes, right? An API or an API provider is always very proud of number of nines in availability. This API is now up for 99.999999%. But I feel like we're kind of ignoring or we're taking the front-end layer not as seriously. And I think that the entire application that we are building should be tested at all the time. And it would be nice if we could have the same statistics for account creation, for account login, for account update and account deletion. It would be great to see that, hey, my account creation tested with a browser works 99.999% of the time. And I think that would just be a nice way if we're, or a nice result if we would get there. And honestly, the technology that we use really doesn't matter here. If you use Cypress, use Playwright, use Puppeteer, but really just test all your properties because I discover, unfortunately, enough bugs out there on the UI layer. And I think we can do better as an industry, but I probably don't have to tell you that because we're here at TestJS Summit. So with that, I think that end-to-end monitoring should really be your safety net and your future self would thank you. My future self already does that and I'm running low on battery now. So thank you everybody for listening. If you want to reach out, my handle is, my name, Stephan Juedis. And if you have any questions, I'm always up for them. So with that, I talk to you very, very soon. ♪♪♪