Testing Web Applications with Playwright

♪♪♪ Testing web applications with Playwright. It's time to play your tests right. Hi, everyone. My name is Debbie O'Brien. I'm a Senior Program Manager at Microsoft, and I'm basically working on advocating for Playwright. I'm an international speaker, teacher, YouTuber, open-source contributor, and you can follow me on Twitter, devs underscore O'Brien, to see me running and cycling and doing all sorts of sports across the island of Mallorca in Spain. So we're here to talk about testing, so let's get straight to it. Playwright testing, reliable end-to-end testing for modern web apps. Works on any browser. It doesn't matter what platform you're on. Any browser, any platform, one API. Tests run in full isolation. That means one test is here with a browser context, another test is here. They don't leak things to each other, right? So everything here is in full isolation from the other test here. Fast execution. We've got powerful tooling, which I'm going to talk about a lot in this talk, and multi-language, which means you can run your tests in C Sharp, in Java, in Python, or in JavaScript and TypeScript. I'm going to show you TypeScript today, but you can choose any of the other languages that you want. So I'm going to show you the VS Code extension, because I absolutely love the integration with VS Code. It's very, very cool, but don't worry. You can still use Visual Studio. You can still just use a terminal or whatever you prefer, but let me show you the extension. So look for the Playwright test for VS Code by Microsoft extension. And then basically in your command panel, just write install Playwright and click on that. And what you get is, now you get to select the browsers and you can choose Chromium, Firefox, or WebKit. You can choose all of them or just one of them. And later you can configure it. So don't worry too much if you're not sure which you want. There's an other option there, use JavaScript. If you don't want to use TypeScript, it's TypeScript by default, but you can use JavaScript if you prefer. And then also the GitHub Actions, by clicking that box, we're going to get a GitHub Actions, and it's going to basically work out of the box without you having to configure anything. You'll be able to run your tests on GitHub, on CI, in every push and pull request. So once you've installed, you're going to get this. You'll get that GitHub folder there with the actions inside it, and you get a test folder with an example.spec.ts file. That will have a very small test in there, which we're going to go through today. And then there's a test example file, which has a to-do demo app, and you can have a look at that in more detail. And then the Playwright config, in case you wanted to later add or remove some of those browsers or put testing for mobile Safari, or set up a dev server to run your application on localhost 3000 before you run your tests, et cetera. You can do all that in the config, but you very rarely need to touch the config because everything just works. So when you want to run tests, you can run all tests. You can run a set of tests, a single test, and tests run in parallel without you having to do anything, automatically runs in parallel, which makes them super fast. So I'm going to run the test using the VS Code extension. Again, you could use the terminal and just put npx playwright test, but here on line three, you'll see this lovely green triangle. So I'm going to click on that, and then basically it's going to go down through each line and it's going to run the test. Now you can see it opened up a browser window there for me, and I was able to see everything that's going on. I'll just play that again. You can see super fast, right? That's because I have this show browser ticked, and you can uncheck that if you want to, if you do not want to see the browser. If you want to see the browser, make sure that that is checked. So live debugging, if you have any errors in your tests, I mean, you never have errors, right? But if you do have any bugs or anything, you have live debugging and error messaging in VS Code itself. So again, we have our test here in VS Code, and I'm just going to like break this here, right? So get by text star, I'm going to run that test, it's going to break, and it says, error strict mode violation. So playwright saying, hey, look, there's one, two, three elements there, and you can have the whole log. Now, if you don't want to read the log or don't understand the log, you could basically just set a break point and you could run that test again. So you can kind of see it. So we're running it in debug mode, and we can now go through that and see what's happening. So at this point in time, playwright saying, hey, look, get by text star, resolve to three elements, here's one of them, here's the other one, and down the bottom, we've got another one that has the word star inside that getting started text. So playwright doesn't know which one you want it to choose. So it's saying, hey, this is, you know, this is breaking the strict violation. So you can play around and live debug this and say, okay, what if I put get star? Get star only resolves to one, so that would be okay. That would work. Again, we can kind of like go back into here and play around and put something like, imagine we put playwright, right? We put playwright, there's like way too many, okay? So it's kind of say, hey, there's like 12 of these. This is gonna be really, really difficult selector to choose from. So yeah, so basically going through this, you're able to kind of like live see what's going on on the page, what playwright's trying to select and see, and you'll be able to then, you know, make sure that those selectors are much better and much more precise. And then you can just like continue, add more debugging, more breakpoints, or just, you know, step through that test, stop it, play it, restart it again and again, et cetera. Now locators, what are locators? So locators are a way to find elements in the page at any moment with built-in auto-weighting and retry ability. Yes, built-in auto-weighting. That means you don't have to do anything. Playwright's gonna wait automatically for that locator to be ready. You don't have to set timeouts or do anything like that. So locators look like this. They're strict by default. So it's page.locator, data test ID equals directions.click, for example. That's if you're using IDs. Now, even better than that, let me show you something else because playwright now has new APIs inspired by testing library. As you can see by this tweet from someone in the community, very, a lot of excitement there. So we've got like, you know, get by text, get by role, get by label, get by placeholder, get by alt text and get by title. And just to show you that in kind of more detail, you can see here. So await page, get by label, username, Phil John. Await page, get by label, password, Phil secret password. Await page, get by role, button, name, sign in. And then expect the page, get by text, welcome John to be visible. So those make sure you're using our new locators, get by label, get by role, get by text, et cetera. And in case you're kind of wondering like, oh my gosh, this is like so difficult. How am I gonna, you know, know which selectors to pick? How am I gonna know which ones? We have a pick selector option in VS Code. Now, if you're not using VS Code, you can also use the terminal. You can basically launch the inspector and you'd be able to do the same thing. So let me show you. So here we have our page. And as we hover over it, after clicking that pick selector, we now get by role heading. We can click that heading there. And inside now, our pick selector box, we've get by role, heading, name, installation. And now I can just like, press enter. It's gonna go into my clipboard and I can now go into my code and basically just paste that into my code. And now I've got my new selector set up and I'm able to kind of just use that. So I can go along here again and I can hover over and see, get by role link. This is name equals how to install PlayRide. Let's check over here, get by role link. This is lovely, get by role button. And this is name, switch between dark and light mode. So that's the name of that one. And again, hovering over each item is gonna give us the selector that we want. So it's a nice easy way of kind of seeing what selectors are available to you and easily able to just like copy that and put that into your code and use it. So yes, pick selector box there. Make sure you play around with that and have a lot of fun picking your selectors. So make sure you're using web first assertions. We have a lot of assertions. We've got to be checked, to be disabled, to be editable, to be empty, to be enabled, to be focused, to be hidden, to be visible, to contain text, to have attribute, to have class, to have count, to have CSS, to have ID, to have JS, to have text, to have titles, to have URL and so many more, right? So I should be a rapper, right? Yeah, so they're all your assertions. So that's what's gonna get a few of that auto weighting. So make sure using those assertions to make sure you get that. Now, let me introduce you to CodeGen. CodeGen is my friend. I love CodeGen. CodeGen is amazing. CodeGen will generate tests by recording your actions. Now it's not perfect, you know, but it's an amazing way to get started and to really help you kind of ease into testing. If you're new to testing, especially, this is a really good way to get started. So let me show you what CodeGen does. Now, again, I'm gonna use the VS Code extension, but you could totally just use the terminal and launch up that inspector and be able to use it from there. So from VS Code, I'm gonna go into that testing sidebar, I'm gonna click record new, and what it's doing is it gives me a browser window, but at the same time, it gives me a test1spec.ts file. It's importing tests from Playwright Tests and it's starting off that test block. Now I can go to any URL, and I'm just gonna test the Netflix website for now. And I come along here and it's got a wait page, go to Netflix, this is great. Now I'll click on this button accept, and you've got this lovely get by role button name accept dot click, so fantastic. And I can continue to carry on and do things. So I'm kind of like clicking on this logo here because I wanna make sure that logo is there. The user might not click on that logo, right, but I'm just clicking on it so it records that action, and I'll later change that to an assertion. Again, click on that heading, and that's not normally a click event, but I wanna click on that so that it's there, and then I'll turn that into an assertion later. Now let's do something fun. Let's go down to the help center. We open up a new page here, and let's click on something. Let's say can't sign into Netflix. Okay, this looks great. We'll click on that title, make sure that is there, lovely, that's perfect. And was this article helpful? Let's click yes, and we'll just check the thanks for your feedback. Did that text exist? That's great. Again, we'll change that later in assertion, and let's start the live chat. Now we've clicked on the live chat, and what do we wanna do? We don't want any of those, so let's tell us what your issue is. Let's click on that, and great. We're now chatting with the Netflix help center. Hello, what can we help you with today? Lovely. Let's just send a reply in there. Let's say testing your chat, and let's send that in. So now I'm in an iframe. Testing the iframe on the website. I wanna chat with an agent. Lovely. Can we sign in? No, let's not sign in. Someone will be with you shortly. Great. And the wait time now is one minute, so we can check that that is there, and let's see what happens. So we are now interacting with an iframe. We're able to easily test that iframe, as you can see, and make sure that the chat works as one would expect it to work. So I'm now chatting with Marcial, so thank you, Marcial, for replying there to me, and I'm just gonna say, hey, I'm just testing the chat. Thanks, Marcial. And we'll send that in there, and then let's not bother the people at Netflix any longer. So let's end that chat. We'll press End, click that button. Yeah, we wanna end it, and email me a transcript of this chat. That sounds really cool, actually. Let's do that. So we'll click on that, and we'll put in our email. I'm just gonna put Debbie and press Send, because I don't wanna put my email, really. And ooh, we've got a, we've sent this transcript to the chat, to the email, so that probably shouldn't have. So we can go and fix that code, right, based on how we're finding things. As you can see, I'm clicking around, and I'm even finding bugs as I'm clicking around and writing the test, and I'm not even actually writing any code. So cool. Back in the footer, we can open up another domain. I'm now testing another domain. I'm testing this fast.com from the Netflix footer. Check that that image is there. Let's click on that button, that Show More Info button, and make sure that the client is there. So again, I'm able to check over multiple domains. I'm able to check iframes, and all my code is written for me, as you can see. And I didn't even have to open any code, right? Now, obviously, we're gonna have to change some of this, which we'll go through in a second, but you can see there's my iframe being tested. You can see my other page is being tested, and everything here is written for me, bang, right there in VS Code. And I can just go along and make sure that works by literally clicking that button. That's gonna basically run up the browser. It's gonna go through here, and you can see it started up the chat, and we'll go to that one-minute wait, and look, Playwright is gonna wait for that to be available. Bang, it's done, it's there. I've got Shania this time. Everything works. Go into the next domain, testing that that works, and up comes my little button. It's gonna wait for that button. It's waiting, it's waiting, it's not there yet. Once that comes up, bang, I've got there, and everything is as expected. So no timeouts had to be set there. We have everything working. So again, we should be using Web Assertions, and you saw I clicked some of those headings. So then I went back over this test, and I improved it. So I changed some of them, like the logo, to be an assertion, so expect it to be visible, and I was able to look at these page, get by role heading, if it's a get by role heading, and I wanna make sure that that is visible. Get by role link, yes, that's a click event. That's perfect. Again, the next one is the URL. That's good. Get by page link, click, perfect, and we can go down through the test, and every time we see a get by role heading, that one, we just want it to be an assertion, so we can easily then modify each one, which makes it a lot easier, kind of, to write your tests, right? You could totally write this completely all on your own without using CodeGen, if that's what you're into, but I just find using CodeGen a lot easier for getting started. So let's talk about show reports. So when you've got your tests, this is, at the moment now, I need to use the terminal for this. I can't use the VS Code extension. So I wanna run my test first, npx playwright test, and then I wanna show report, npx playwright show report. Now, what this gives me is a whole report of my tests, and I can see, whoa, it failed on Firefox, it passed on Chromium and WebKit, and then I could go in and inspect what happened in Firefox. I could see a whole log of it, but I wanna show you the trace, because the trace is so much more fun. It's that little icon there, the three little bars, so I can click on that, and it will open up the trace, or I can go into detail of that, and you'll see at the very, very bottom, the trace as well. When I click on the trace, what it will open up and show, well, first of all, let me show you how you can run the traces, right? You can record a trace for each test locally using npx playwright test dash dash trace on. Now, normally the trace is run on CI on the first retry, so if your test fails on the first retry, a trace is recorded for you. You don't have to do anything to set that up, it's all done, but if you wanted to do it locally, you would do npx playwright test dash dash trace on, and then basically open up the report, and then click on that, and you'll see the trace. So this is the trace here. Basically, what the trace is doing is it's recording every single thing that happened during that test, so you'll be able to then go along that timeline, and in your own time, go back and forth, and kind of really see what's going on. So these are image snapshots of every single action. You can see each one is highlighted with different colors, locator click frame, that was the iFrame, so every kind of event is highlighted there, and you can basically go through that, scroll across, go back, kind of gently see what was happening. You can see there my email address, Debbie. You can see I sent, you can see there's no address added in there. You'll be easily able to point out when something went wrong and so that someone is gonna be able to fix it. They can really see, oh, look what happened there. Oh, I can see, I can really kind of slow it down and speed up and kind of find out. Now, if you don't wanna use the timeline, you can also go through the actions and stuff. Now, the actions, what it's gonna do when I click here in the middle, this is a DOM snapshot, not an image snapshot, so this is even better, and you can see the click event, the action, the little red dot is highlighted there in every single action, and we'd be able to see what happened before, what happened after, and we can go through those actions and you can see which one was clicked, the text, thanks for the iframe, we're now in the iframe, what's going on in that iframe, and every single one, the locator clicked there, they expect it to be visible, the iframe, and then over on the other side, you can see the call, the console, the network, and the source, so you can see at each time during each action, what's going on in that call, the duration, the wait time, which selectors, if it's strict, what value, the whole log, waiting for the iframe, the selector is off to visible, waiting for the element to be visible, element is visible, enabled, and editable, and you saw I actually wrote in there because it is an actual DOM snapshot rather than an image snapshot, so that means you can also open the Chrome DevTools, you can expect the CSS in there, you can really kind of understand what's going on, and you can see then you've got like the different network requests are highlighted there, so one of them we had one network request there, and as you go along, you can see everything that's going on as well in the source code as well, so you can see the source for your test. Now, this is a PWA, which basically, you can then send to someone else, so they can then see the source code rather than actually just having to open up the actual code itself, so the metadata's also there, you see if it's mobile, et cetera, so this is a really, really good way of debugging your tests, I love it also when I'm building, but yeah, it's very, very cool, so make sure you check out the Trace Viewer. Now, you can also run your tests on CI very easily with GitHub Actions, this is what you're gonna get, that's already included for you once you tick that box, which is the GitHub Actions, and once that's happened, you know your tests are either gonna fail or they're gonna pass, you're gonna have it red, you're gonna have it green, and if they're red, especially you wanna kinda like find out what's going on, so you can run, look into those actions, go along, check out the log, and you can also see that Playwright report there, I showed you the report locally, that's where you get the report on CI, which you can then download and use Playwright to then serve that up, and you'll be able to click on that and trace and kind of visualize that trace and see what happened in CI, why did that fail, and debug that and improve your tests. So with Playwright testing, you have the VS Code extension, your code gen, my friend, is gonna auto-generate the tests for you, you can run your tests in debug mode, you've got locators and frame locators, so you can test iframes, you can see reports, you can open up the trace viewer and really see a full trace of those, you can still do screenshots and videos if you like, but I just love the trace viewer, so that's by default, but if you prefer videos, totally up to you, you can also do that. Auto-wait built in for all actions, so there's no need to set timeouts, tests run in parallel for free without you having to do anything, which makes it super fast, it runs across all browsers and devices, so emulates mobile devices as well, intercepts requests, follows all, redirects and bypasses cores and manages cookies for you. So, who's using Playwright? Absolutely everybody, I mean, check out that chart. Playwright is open source and free, it's by Microsoft, and we absolutely love GitHub stars, so make sure you give us a star. And these are our amazing ambassadors, check them out, they're on our community tab, ambassador page, and they're creating amazing content on Playwright, so do connect with them and check out their video and courses and blog posts and et cetera, and make sure you're following them. But seriously, I just have one question for you. Are you ready? Are you ready for the question? Are you ready to Playwright? I wanna hope to see you all, basically, in the Playwright community, creating great tests, let's improve the testing scene in general, let's write better code, let's test that code, and let's just have some fun and make testing easy and fun, that's the goal. Thank you very much for your time, check out our docs, check out our Twitter, check out our YouTube and our community Slack channel, and if you wanna reach out for anything, you can find me, devs underscore Brian on Twitter, my website, debbie.codes, and I really look forward to seeing so many more of you in the Playwright community. Thanks everyone, bye. And that was such a power-packed talk by Debbie. So before the talk, we asked a question, and now I would like to invite Debbie on the stage with me and we will discuss the Q&A and the poll questions. Debbie, please join me, hi. Woo-hoo, hi. Thank you, Debbie, that was such an amazing talk and actually I learned a lot, so I saw Playwright for the first time and I think I'm gonna try, and so are many of the audiences, of course, after seeing the talk. So we asked the question, is like a Playwright API can be used with all of these, except, and I think 93% of the people say C++. So what do you say, is that the correct answer? I think they did pretty good, it's quite tricky, right? Because it's C+, right? And not C++, right? And if you're not into the C pluses, then you might get confused. But yes, Playwright works on C sharp or we can say.NET, right? Python, also TypeScript, JavaScript and Java. So we have pretty broad, yeah, like a coverage of languages in which you can write Playwright tests. Exactly, which is kind of cool, right? Because I mean, if we work in the same company, right? And now our backend developers who are writing in Java, for example, or.NET, they can continue writing in their language and we're gonna write in TypeScript and you're gonna write in JavaScript because you're afraid of TypeScript, but that's fine. And we'll be able to still use the same testing tool, even though we're using different languages and we'll all understand each other's tests, but we'll all write them in the way that we understand. So it's kind of cool. That's cool. I just, a question which comes out of my mind out of this is that, can you write all the different language testing for the same project or like? Yeah, so you can decide to put your tests with your project. So if you're writing a Node.js project, then obviously if your tests are inside that project and you wanna run it, you're better off running those in Node, but you can also run them outside of your project or you could just set up a dev server in a GitHub Actions to kind of like launch that server. So it all depends on where your server is running to be able to like run those tests or not, but you can totally test outside your application, inside it, whatever works better. Awesome. So there's a question Mark R asks. Yeah, I thought that it would be a question. What are the advantages of using Playwright over Cypress? That's a horrible one. Because you know, Cypress is great. I love Cypress, used to work with Cypress. So what are the advantages over Cypress? I guess what I showed in the talk specifically, that be able to test chat applications, go across multiple domains, go into iFrames without anything else, just it just works parallel for free and probably other things, but let's not go down that road, but yeah. Test your application. I think that those two are pretty nice frameworks. So yeah. So we have a question from Marie. With the new API page.getByTestID, does this require the custom attribute to be data test ID or can it automatically detect other custom attributes such as data-test or something similar? Yeah, of course. So we just set one and then like, if that works for you, perfect. But if not, you could write data, my beautiful tests if you want, right? You can change that attribute to whatever you want. So you can configure that. Awesome, awesome. Thank you. That sounds a pretty nice idea for attribute name for sure. Yeah. So yeah, that's also a question for people who might want to get started with Playwright. Like how can they get started with Playwright? So I recommend you take three minutes out of your day. That's all it's gonna take, three minutes out of your day and literally just go through the getting started guide. I know people go, oh my God, docs, I hate docs. I don't want it, but it's so simple because literally it just takes you through installing. You can use the VS code extension, which I showed. And then you don't even have to write a test because you've already been given a test. Press the play button, it's gonna run, right? Now you're on a minute and a half. So now another 30 seconds to get code gen, create a test just by clicking, bang, you've got that other test, run it. And then you can put up on GitHub actions in another minute, three minutes, test running up on GitHub actions, up on CI, easy. See, three minutes is all it takes, right? Yes. To go from zero to hero in Playwright, yeah. And if you fail, if you fail, come and talk to me. If it doesn't work for you, come and talk to me. I wanna know why, because it should. Awesome, yeah, thank you, thank you. So also we have a question, in your opinion, what is the future of Playwright? The future, the future is that everybody is gonna love testing and they're gonna always be testing the applications. So we never have problems that we click something and it doesn't work, that the user gets frustrated. We're gonna have no frustration in the web anymore because if we make testing easy and fun, more people will test, testing will just happen and it will become a culture, part of the culture, like in the Netherlands, cycling a bike, right? That's a culture. Let's get the whole world cycling bikes. Let's get the whole world riding tests. So yeah, that's what I think the future is. That is an amazing answer, really. Like testing are very common and easy. So everyone gets into it for sure, thank you. Also, so what is like, you have been working for with Playwright for a while, right? So what is your favorite feature of Playwright so far? You know, it's kind of hard because I'm between Cogen and the Trace Viewer. Cogen I love because I never have to think about which of those locators that you need to choose. I don't think I just click, it tells me which ones and then I can look at it and say, yeah, you know, that's good, looks good, perfect. It writes my test for me, that saves a lot of time. And then Trace Viewer, when the test fails, to be able to do that, but I'm just not really getting a lot of fails tests. So I'm gonna say Cogen because it's helping me really get a lot of tests done. When I saw it, like for me also, it was like that. Cogen was like, wow, that sounds super cool. Yeah, so. This is why I have so much time to do sports, right? Because Cogen is writing my tests for me. Now you have a secret of Debbie doing three hours and six hours of sport every day. Yeah, so like I already asked you, but just for the audience again, what's like, how many hours did you do today and what are you going to do next for the sports? Yeah, so I ran to the gym, that was one kilometer. I played a match that was an hour and a half, a power match. And I ran home, that was another kilometer. And in an hour's time, I'm gonna run to the gym for my second match. Got actually two matches, but they're shorter matches. So it'll be another hour and a half. So it'll be about three hours, 20 minutes. That's just amazing. Like the way you like, it really motivates. So on that note, a very other question is how to stay motivated and so energetic? Like how would people be so energetic? Like give us some tips, come on. You literally have to put it in the calendar. I put it, it's in my calendar that I need to go to sport. Cause if not, I'm gonna sit here and I'm gonna code all night long and I'll forget to even have dinner. So literally I put it in, if it's in the calendar, it's gonna happen. If not in the calendar, it doesn't happen. Love it. But thank you. Thank you so much, Debbie, again for this wonderful talk and showing us all about Playwright and answering all the questions. It was really amazing to have you today with us. It was lovely to see you again. And thanks for having me here. And thanks everyone for watching. ♪♪♪