1. Introduction to Playwright Testing
♪♪♪ Testing web applications with Playwright. It's time to play your tests right. Hi, everyone. My name is Debbie O'Brien. I'm a Senior Program Manager at Microsoft, and I'm basically working on advocating for Playwright. I'm an international speaker, teacher, YouTuber, open-source contributor, and you can follow me on Twitter, devs underscore o'brien, to see me running and cycling and doing all sorts of sports across the island of Majorca in Spain.
2. Running Tests and Live Debugging with Playwright
Once you've installed Playwright, you'll have access to the GitHub folder with actions, a test folder with example files, and the Playwright config. Running tests is easy with the VS Code extension, which provides live debugging and error messaging. By setting breakpoints, you can analyze and debug your tests to resolve any issues. Playwright also offers parallel test execution and the option to show or hide the browser window during testing.
So once you've installed, you're gonna get this. You'll get that GitHub folder there with the actions inside it, and you get a test folder, when you example.spec.ts file. That will have a very small test in there, which we're gonna go through today, and then there's a test example file, which has a to-do demo app, and you can have a look at that in more detail.
And then the PlayWrite config, in case you wanted to later add or remove some of those browsers, or put testing for mobile Safari, or set up a dev server to run your application on localhost 3000 before you run your tests, et cetera. You can do all that in the config, but you very rarely need to touch the config, because everything just works.
So when you wanna run tests, you can run all tests. You can run a set of tests, a single test, and tests run in parallel. Without you having to do anything, automatically runs in parallel, which makes them super fast. So I'm gonna run the tests using the VS Code extension. Again, you could use the terminal and just put npx.playwritetest, but here on line three, you'll see this lovely green triangle. So I'm gonna click on that, and then basically, it's gonna go down through each line, and it's gonna run the test. Now, you can see it opened up a browser window there for me, and I was able to see everything that's going on. I'll just play that again. You can see super fast, right? That's because I have this show browser ticked, and you can uncheck that if you want to. If you do not wanna see the browser, if you wanna see the browser, make sure that that is checked.
So live debugging, if you have any errors in your test, I mean, you never have errors, right? But if you do have any bugs or anything, you have live debugging and error messaging in VS Code itself. So again, we have our test here in VS Code, and I'm just gonna like break this here, right? So getByText, I'm gonna run that test, it's gonna break, and it says, error, strict mode violation. So playwright's saying, Hey, look, there's one, two, three elements there, and you can have the whole log. Now, if you don't wanna read the log or don't understand the log, you could basically just set a break point, and you could run that test again, so you can kind of see it. So running it in debug mode, and we can now go through that and see what's happening. So at this point in time, playwright's saying, Hey, look, getByText, resolved to three elements. Here's one of them. Here's the other one. And down the bottom, we've got another one that has the word star inside that getting started text. So playwright doesn't know which one you want it to choose. So it's saying, Hey, this is breaking the strict violation. So you can play around and live debug this and say, Okay, what if I put get star? Get star only resolves to one, so that would be okay. That would work. Again, we can kind of like go back into here and play around and put something like, imagine we put playwright, right? We put playwright. There's like way too many, okay? Hey, there's like 12 of these.
3. Selectors and Locators in Playwright
Selectors in Playwright allow you to choose elements on a page with precision. Playwright provides built-in auto waiting and retry ability for locators, eliminating the need for timeouts. New APIs inspired by testing library, such as getByText, getByRole, and getByLabel, offer even more options. Playwright also offers a pick selector option in VS Code, making selector selection easier. By hovering over elements, you can obtain locators and quickly insert them into your code.
This is gonna be really, really difficult selector to choose from. So, yeah, so basically going through this, you're able to kind of like live see what's going on in the page, what playwright's trying to select and see, and you'll be able to then, make sure that those selectors are much better and much more precise. And then you can just like continue, add more debugging, more break points, or just step through that test, stop it, play it, restart it again, again, et cetera.
Now locators, what are locators? So locators are a way to find elements in the page at any moment with built-in auto waiting and retry ability. Yes, built-in auto waiting, that means you don't have to do anything. Playwright's gonna wait automatically for that locator to be ready. You don't have to set time outs or do anything like that.
So, locators look like this. They're strict by default. So it's page.locator, data test ID equals directions.click, for example, let's say we're using IDs. Now, even better than that, let me show you something else because Playwright now has new APIs inspired by testing library. As you can see by this tweet from someone in the community, a lot of excitement there. So we've got like, you know, getByText, getByRole, getByLabel, getByPlaceholder, getByAltText and getByTitle. And just to show you that in kind of more detail you can see here. So, await page, getByLabel, username, philjohn, await page, getByLabel, password philjohn, password, await page, getByRole, button, name, sign in, and then expect the page, getByText, welcome John to be visible. So those make sure you're using our new locators, getByLabel, getByRole, getByText, et cetera.
And in case you're kinda wondering like, oh my gosh, this is like so difficult. How am I gonna, you know, know which selectors to pick? How am I gonna know which ones? We have a pick selector option in VS Code. Now if you're not using VS Code, you can also use the terminal. You can basically launch the inspector and you'll be able to do the same thing. So let me show you. So here we have our page. And as we hover over it after clicking that pick selector, we now getByRole heading. We can click that heading there. And inside now our pick selector box, we've getByRole, heading, name, installation. Now I can just like press Enter. It's gonna go into my clipboard and I can now go into my code and basically just paste that into my code. And now I've got my new selector set up and I'm able to kind of just use that. So I can go along here again and I can hover over and see getByRole link. This is name equals how to install PlayRide.
4. Selectors, Assertions, and Code Gen in Playwright
Playwright provides a variety of selectors for choosing elements on a page. Web-first assertions, such as to be checked, to be disabled, to be editable, and many more, ensure accurate testing. Code Gen is a helpful tool for generating tests by recording actions. It's a great way to start testing, especially for beginners. Using the VS Code extension or the terminal, you can easily record tests and generate code.
Let's check over here, getByRole link. This is lovely, getByRole button. And this is name, switch between dark and light mode. So that's the name of that one. And again, hovering over each item is gonna give us the selector that we want. So it's a nice easy way of kind of seeing what selectors are available to you and easily able to just like copy that and put that into your code and use it. So yes, pickSelectorBox there, make sure you play around with that and have a lot of fun picking your selectors.
So make sure you're using web-first assertions. We have a lot of assertions. We've got, to be checked, to be disabled, to be editable, to be empty, to be enabled, to be focused, to be hidden, to be visible, to contain text, to have attribute, to have class, to have count, to have CSS, to have ID, to have JS property, to have text, to have titles, have your own and so many more. Right? So I should be Robert, right? Yeah, so they're all your assertions. So that's what's gonna get a few that auto-weighting. So make sure you're using those assertions to make sure you get that.
Now, let me introduce you to Code Gen. Code Gen is my friend. I love Code Gen. Code Gen is amazing. Code Gen will generate tests by recording your actions. Now it's not perfect, you know, but it's an amazing way to get started and to really help you kind of ease into testing if you're new to testing especially. This is a really good way to get started. So let me show you what Code Gen does. Now, again, I'm gonna use the VS Code extension, but you could totally just use the terminal and launch up that inspector and be able to use it from there. So from VS Code, I'm gonna go into that testing sidebar, I'm gonna click record new. And what it's doing is it gives me a browser window, but at the same time, it gives me a test1spec.ts file. It's importing tests or playwright tests and it's starting off that test block. Now I can go to any URL. And I'm just gonna test the Netflix website for now. And I come along here and it's got a wait page, go to Netflix, this is great. Now I'll click on this button accept and you've got this lovely get by roll button, name accept dot click. So, fantastic.
5. Testing Help Center and Live Chat
Continuing actions by clicking on logo and heading. Testing Help Center features, live chat, and iframe interaction. Chatting with Martial and ending the chat.
And I can continue to carry along and do things. So I'm kind of like clicking on this logo here because I want to make sure that logo is there. The user might not click on that logo, right? But I'm just clicking on it so it records that action and I'll later change that to an assertion. Again, click on that heading and that's not normally a click event but I want to click on that so that it's there and then I'll turn that into an assertion later.
Now let's do something fun. Let's go down to the Help Center. We open up a new page here and let's click on something, let's say can't sign into Netflix. Okay, this looks great. We'll click on that title, make sure that is there, lovely, that's perfect. And was this article helpful? Let's click yes. And we'll just check the thanks for your feedback, did that text exist? That's great. Again, we'll change that later an assertion and let's start the live chat. Now, I clicked on the live chat and what do we want to do? We don't want any of those so let's tell us what your issue is. Let's click on that and great, we're now chatting with the Netflix Help Center. Hello, what can we help you with today? Lovely. Let's just send a reply in there, let's say, testing your chat and let's send that in.
So now I'm in an iframe, testing the iframe on the website. I want to chat with an agent, lovely. Can we sign in? No, let's not sign in. Someone will be with you shortly, great. And the wait time now is one minute so we can check that that is there and let's see what happens. So we are now interacting with an iframe. We're able to easily test that iframe, as you can see, and make sure that the chat works as one would expect it to work. So I'm now chatting with Martial, so thank you Martial for replying there to me and I'm just gonna say, hey, I'm just testing the chat. Thanks Martial. And we'll send that in there and then let's not bother the people at Netflix any longer. So let's end that chat. We'll press end, click that button. Yeah, we want to end it and email me a transcript of this chat. That sounds really cool actually.
6. Testing with Playwright and Generating Code
Clicking around and finding bugs without writing code. Testing multiple domains and iframes. Playwright generates code for testing. No timeouts needed. Use web assertions and improve tests.
Let's do that. So click on that and we put in our email. I'm just gonna put Debbie and press send cause I don't want to put my email really. And oh, we've got a, we've sent this transcript to the chat, to the email, so that probably shouldn't have. So we can go and fix that code, right? Based on how we're finding things.
So as you can see, I'm clicking around and I'm even finding bugs as I'm clicking around and writing the test and I'm not even actually writing any code. So cool. Back in the footer, we can open up another domain. I'm not testing another domain. I'm testing this fast.com from the Netflix footer, check that that image is there. Let's click on that button, that show more info button and make sure that the client is there. So again, I'm able to check over multiple domains. I'm able to check iframes and all my code is written for me, as you can see. And I didn't even have to like open any code, right?
Now, obviously we're going to have to change some of this, which we'll go through in a second, but you can see there's my iframe being tested. You can see my other page is being tested and everything here is written for me, bang, right there in VS Code. And I can just go along and make sure that works by literally clicking that button. That's gonna basically run up the browser. It's gonna go through here and you can see it started up the chat and we'll go to that one minute wait and look, Playwright is gonna wait for that to be available. Bang, it's done, it's there. I've got chunia this time. Everything works, going to the next domain, testing that that works and up comes my little button. It's gonna wait for that button. It's waiting, it's waiting, it's not there yet. Once that comes up, bang, I've got there and everything is as expected. So no timeouts had to be set there. We have everything working. So again, we should be using web assertions and you saw I clicked some of those headings. So then I went back over this test and I improved it. So I changed some of them like the logo to be an assertion. So I expect it to be visible and I was able to like look at these page, get by roll heading.
7. Using Code Gen and Analyzing Traces
When writing tests, using code gen can make it easier to get started. Show reports provide a comprehensive view of test results, including logs and traces. Traces record every action during a test, allowing for detailed analysis and troubleshooting. Traces can be run locally using the command npx playwright test --trace on.
If it's a get by roll heading and I wanna make sure that that is visible. Get by roll link, yes, that's a click event that's perfect. Again, the next one is the URL that's good. Get by page link, click, perfect. And we can go down through the test and every time we see a get by roll heading, that one, we just want it to be an assertion so we can easily then modify each one which makes it a lot easier kind of, you know, to write your test, right? You could totally write this completely all on your own without using code gen if that's what you're into but I just find using code gen a lot easier for getting started.
So let's talk about show reports. So when you've got your tests, this is at the moment now, I need to use the terminal for this. I can't use the VS Code extension. So I wanna run my test first, npx playwright test, and then I wanna show report. npx playwright show report. Now what this gives me is a whole report of my tests and I can see, whoa, it failed on Firefox. It passed on Chromium and WebKit and then I could go in and inspect what happened in Firefox. I can see a whole log of it but I wanna show you the trace because the trace is so much more fun. It's that little icon there, the three little bars. So I can click on that and it will open up the trace or I can go into detail of that and you'll see at the very, very bottom, the trace as well. When I click on the trace, what it will open up and show, well, first of all, let me show you how you can run the traces, right? You can record a trace for each test locally using npx playwright test dash dash trace on. Now normally the trace is run on CI on the first three tries. If your test fails on the first retry a trace is recorded for you. You don't have to do anything to set that up, it's all done. But if you wanted to do it locally, you would do npx playwright test dash dash trace on and then basically open up the report and then click on that and you'll see the trace. So this is a trace here. Basically what the trace is doing is it's recording every single thing that happened during that test. So you'll be able to then go along that timeline and in your own time, go back and forth and kind of really see what's going on. So these are image snapshots of every single action. You can see each one is highlighted with different colors, locator, click frame that was the iframe. So every kind of event is highlighted there and you can basically like go through that, scroll across, go back, kind of gently see what was happening. You can see there my email address, Debbie, you can see I sent, you can see there's no address added in there. You'll be easily able to point out when something went wrong and so that someone is gonna be able to fix it. They can really see, oh look what happened there. Oh I can see, I can really kind of, slow it down and speed up and kind of find out.
8. Debugging Tests and Running on CI with Playwright
Now you can debug your tests with the Trace viewer, which provides a comprehensive view of test actions, including DOM snapshots, network requests, and source code. Additionally, you can run tests on CI using GitHub Actions and analyze the Playwright Report to debug and improve your tests. Playwright testing offers the VS Code extension and Code Gen for auto-generating tests.
Now if you don't want to use the timeline, you can also go to the actions and stuff. Now the actions, what it's gonna do when I click here in the middle. This is a dom snapshot, not an image snapshot, so this is even better. And you can see the click event, the action, the little red dot is highlighted there in every single action. And we'd be able to see like what happened before, what happened after. And we can go to those actions and you can see which one was clicked.
The text thanks for the iframe, we're now in the iframe. What's going on in that iframe and every single one the locator clicked there, they expect it to be visible, the iframe. And then over in the other side, you can see the call, the console, the network, and the source. So you can see at each time during each action, what's going on in that call, the duration, the wait time, which selectors. If it's strict, what value, the whole log waiting for the iframe, the selector is also visible, waiting for the element to be visible, element is visible, enabled and editable. And you saw I actually wrote in there because it is an actual DOM snapshot rather than an image snapshot. So that means you can also open the Chrome DevTools, you can expect the CSS in there, you can really kind of understand what's going on. And you can see then you've got like the different network requests are highlighted there. So one of them we had one network request there, and as you go along, you can see everything that's going on as well in the source code as well. So you can see the source for your test.
Now, this is a PWA, which basically you can then send to someone else so they can then see the source code rather than actually just, you know, having to open up the actual code itself. So the metadata is also there, you see if it's mobile, etc. So this is a really, really good way of debugging your test, I love it also when I'm building, but yeah, it's very, very cool. So make sure you check out the Trace viewer.
Now you can also run your test on CI very easily with GitHub Actions. This is what you're gonna get that's already included for you once you tick that box, which is the GitHub Actions. And once that's happened, you know your test is either gonna fail or they're gonna pass. You're gonna have it red, you're gonna have it green. And if they're red, especially you wanna kind of like find out what's going on. So you can run, look into those actions, go along, check out the log, and you can also see that Playwright Report there. I showed you the report locally, that's where you get the report on CI which you can then download and use Playwright to then serve that up. And you'll be able to click on that and trace and kind of, you know, visualize that trace and see what happened in CI, why did that fail? And you know, debug that and improve your test. So with Playwright testing, you have the VS Code extension. You got Code Gen, my friend, is gonna auto-generate the test for you.
9. Benefits of Playwright and Community Engagement
You can run tests in debug mode, with locators and frame locators for testing iframes. The Trace viewer provides a full trace of actions, and you can also take screenshots and videos. Auto-wait is built-in, eliminating the need for timeouts. Tests run in parallel across all browsers and devices, with interception of requests, following redirects, and cookie management. Playwright is used by everyone, is open source, and free. Connect with our ambassadors for amazing content and get involved in the Playwright community. Are you ready to join us in improving testing and having fun? Check out our documentation, Twitter, YouTube, and Slack channel.
You can run your test in debug mode. You've got locators and frame locators. So you can test iframes. You can see reports, you can open up the Trace viewer and really see a full trace of those you can still do screenshots and videos if you like, but I just love the trace viewer.
So that's by default, but if you prefer videos, totally up to you, you can also do that. Auto-wait built in for all actions. So there's no need to set timeouts. Tests run in parallel for free without you having to do anything which makes it super fast. It runs across all browsers and devices. So emulates mobile devices as well. Intercepts requests, follows all redirects and bypasses cores and manages cookies for you.
So who's using Playwright? Absolutely everybody. I mean, check out that chart. Playwright is open source and free. It's by Microsoft and we absolutely love GitHub stars. So make sure you give us a star. And these are amazing ambassadors. Check them out. They're on our community tab ambassador page and they're creating amazing content on Playwright. So do connect with them and check out their video and courses and blog posts and et cetera and make sure you're following them.
But seriously, I just have one question for you. Are you ready? Are you ready for the question? Are you ready to playwrite? I want to hope to see you all basically in the playwright community creating great tests. Let's improve the testing scene in general. Let's write better code. Let's test that code and let's just have some fun and make testing easy and fun. That's the goal. Thank you very much for your time. Check out our docs. Check out our Twitter. Check out our YouTube and our community Slack channel.
Q&A and Advantages of Playwright
And if you want to reach out for anything, you can find me, Debs underscore Brian on Twitter. My website Debbie dot codes. And I really look forward to seeing so many more of you in the playwright community. Thanks everyone, bye.
And that was such a power packed talk by Debi. So before the talk, we ask the question and now I'd like to invite Debbie on the stage with me and we will discuss the Q and A and the poll questions. Debbie, please join me. Hi. Woohoo. Hi. Thank you Debbie. That was such an amazing talk and actually I learned a lot. So I saw playwright for the first time and I think I'm gonna try and so are many of the audiences, of course, after seeing the talk.
Advantages and Future of Playwright
Advantages of Playwright over Cypress include testing chat applications, multiple domains, and iframes. The new API page.getByTestId allows custom attribute configuration. Getting started with Playwright is simple and can be done in three minutes using the provided guide and VS Code extension. The future of Playwright is to make testing easy and fun, creating a culture of testing for frustration-free web experiences.
So what are the advantages over Cypress? I guess what I showed in the talk specifically, be able to test chat applications, go across multiple domains, go into Iframes without anything else, just it just works. Parallel for free and probably other things, but let's not go down that road. But yeah, test your applications. I think that those two are pretty nice frameworks, so yeah.
So we have a question from Murray, with the new API page.getByTestId, does this require the custom attribute to be data test ID or can it automatically detect other custom attributes such as data-test or something similar? Yeah, of course, so we just set one and then if that works for you, perfect. But if not, you could write data my beautiful tests if you want, right? You can change that attribute to whatever you want so you can configure that. Awesome, awesome, yeah, thank you. That sounds a pretty nice idea for attribute name for sure.
Yeah, so yeah, that's also a question for people who might want to get started with PlayWrite. Like how can they get started with PlayWrite? So I recommend you take three minutes out of your day. That's all it's gonna take, three minutes out of your day and literally just go through the getting started guide. I know people go, oh my God docs, I hate docs. I don't want it but it's so simple because literally it just takes you through installing. You can use the VS Code extension, which I showed. And then you don't even have to write a test because you've already been given a test. Press the play button, it's gonna run, right? Now you're on a minute and a half. So now another 30 seconds to get code gen, create a test just by clicking. Bang, you've got the other test run it, and then you can put up on GitHub Actions in another minute, three minutes, run test running up on GitHub Actions up on CI easy. See, three minutes is all it takes, right? To go from zero to hero in playwright, yeah. And if you fail, if you fail, come and talk to me. If it doesn't work for you, come and talk to me. I wanna know why because it should. Awesome, yeah, thank you, thank you.
So also we have a question, in your opinion, what is the future of playwright? The future? The future is that everybody is gonna love testing and they're gonna always be testing their applications. So we never have problems that we click something and it doesn't work, that the user gets frustrated. We're gonna have no frustration in the web anymore because if we make testing easy and fun, more people will test. Testing will just happen and it will become a culture, part of the culture. Like in the Netherlands, cycling a bike, right? That's the culture. Let's get the whole world cycling bikes. Let's get the whole world riding tests.
Favorite Feature of Playwright
The future of Playwright is bright, with testing becoming common and easy. The Codegen feature is a favorite among users, as it automatically generates tests, saving time and effort. The Trace viewer is also appreciated for troubleshooting failed tests. With Codegen doing the heavy lifting, Debbie has more time for sports, spending around three hours and twenty minutes on physical activities.
So yeah, that's what I think the future is. That is a amazing answer. Really, like testing are very common and easy. So everyone gets into it for sure. Thank you.
Also, so what is like, you have been working for playwright for a while, right? So what is your favorite feature of playwright so far? You know, it's kind of hard because I'm between Codegen and the Trace viewer. Codegen I love because I never have to think about which of those locators that you need to choose. I don't think I just click, it tells me which ones and then I can look at it and say, yeah, you know, that's good, looks good, perfect. It writes my tests for me, that saves a lot of time. And then Trace viewer, when the test fails, to be able to do that. But I'm just not really getting a lot of fails tests. So I'm going to say Codegen because it's helping me really get a lot of tests done. But then when I saw it like for me also, it was like that Codegen was like, wow, that sounds super cool. Yeah. Yeah. So, yeah. This is why I have so much time to do sports, right? Because Codegen's writing my tests for me. See now you have secret of Debbie doing three hours and six hours of sport every day. Yeah, so like I already asked you but just for the audience again, what's like how many hours did you do today and what are you going to do next for the sports? Yeah, so I ran to the gym. That was one kilometer. I played a match. That was an hour and a half power match. And I ran home. That was another kilometer. And in an hour's time I'm going to run to the gym for my second match. I've got actually two matches, but they're shorter matches. So I'll be another hour and a half. So I'll be about three hours, 20 minutes. That's just amazing. Like the way you like it really motivates.
Staying Motivated and Thank You
To stay motivated and energetic, put activities in your calendar. Debbie thanks the audience for the talk and sharing information about Playwright.
So on that note, I'm very, and the other question is how to stay motivated and so energetic. Like how would people be so energetic? Like give us some tips, come on. You literally have to put it in the calendar. It's in my calendar that I need to go to sport. Cause if not, I'm going to sit here and I'm going to code all night long and I'll forget to even have dinner. So literally I put it in, if it's in the calendar, it's going to happen. If it's not in the calendar, it doesn't happen.
Love it. But thank you. Thank you so much Debbie again for this wonderful talk and showing us all about Playwright and answering all the questions. It was really amazing to have you today with us.