The modern web platform is continuously evolving. Today's web apps are more sophisticated than ever before and testing for the modern web requires modern primitives. In this talk, we will cover how Playwright is uniquely enabling web developers to ship faster and more confidently.
Testing for the Modern Web with Playwright
AI Generated Video Summary
Arjun introduces Playwright, a library for cross-browser end-to-end testing of web applications. The challenges in modern web testing include the evolving web platform and reactive frameworks. Playwright provides capable, reliable, and fast automation, with the ability to capture events and enhance test reliability. It uses browser contexts for faster execution and supports multiple languages and frameworks. Playwright is used by companies like VS Code, Bing, Adobe, and material UI, and offers cross-browser support and an improving roadmap.
1. Introduction to Testing the Modern Web
Hi, my name is Arjun, and I'm excited to be here at Test.js and talk to you about testing. I'm a program manager at Microsoft where my team is building Playwright. Playwright is a new library for cross-browser end-to-end testing of web applications. In this talk I'm going to talk to you about testing the modern web. My team and I have interacted with hundreds of testers and developers that are building modern web applications, and we've shaped Playwright for their needs.
Hi, my name is Arjun, and I'm excited to be here at Test.js and talk to you about testing. I'm a program manager at Microsoft where my team is building Playwright. Playwright is a new library for cross-browser end-to-end testing of web applications.
In this talk I'm going to talk to you about testing the modern web. My team and I have interacted with hundreds of testers and developers that are building modern web applications, and we've shaped Playwright for their needs. I'm also going to show you the core ideas behind Playwright and hopefully convince you why Playwright is the right testing tool if you're building a modern web application.
Let's jump right in. Let's start by talking about the modern web platform. The modern web platform is more capable than ever before. Web browsers have matured significantly over the last few years. They run on every device we have, including our desktop, our phones, and smart TVs. There's also been a continuous inflow of new features on the web platform. For example, the web can now detect your current location, it can connect to Bluetooth devices, and also interact with your file system. The pace of innovation in the web platform has been incredible.
2. Challenges in Modern Web Testing
These changes in the web platform have powered innovation in the application world. Web applications of today are full-blown applications with rich and responsive interactions. Teams are struggling to automate all parts of their application due to the evolving web platform and the challenges posed by reactive frameworks. This has resulted in end-to-end tests slowing teams down instead of helping them move faster. Modern web testing needs to be capable, reliable, and fast to keep up with the changing web landscape.
With these changes, we've also seen changes in how teams are operating. Teams of developers and testers are actually moving faster. Lots of teams are building applications today, and there's competitive pressures which are pushing teams to ship faster, because that is how they achieve their business goals. This has led to teams adopting newer workflows like CI, CD, and continuous deployment. And teams are also looking for automated testing solutions so that they can release even faster. Unfortunately, end-to-end testing has not evolved as quickly as the web itself. Let's talk a little bit more about that.
End-to-end testing needs to actually evolve for the changes that we talked about in the modern web. For example, tests need to be able to automate newer platform features that are showing up across browsers. These include capabilities that are available on desktop browsers, capabilities that are available on mobile browsers, and even capabilities that are being powered in smart TVs. Teams are actually struggling to be able to automate all parts of their application because their tests cannot actually cover all the capabilities that they're using. The applications of today are also harder to automate. Tests cannot keep up with how reactive frameworks work. And so teams that are building rich single-page applications find that their end-to-end tests are flaky. They're not able to reliably automate user interactions on their applications. What happens as a result is that end-to-end tests that are actually supposed to help you move faster are actually slowing teams down. Teams end up having to add sleep timeouts or retries to be able to manage reliability issues. Teams are struggling with coverage issues because they're not able to automate all parts of their application. And therefore, end-to-end tests are actually slowing teams down when they actually need to be able to ship faster. These three problems are actually the reason why end-to-end testing has not been able to evolve for the modern web. And these three actually give us really the three pillars of modern web testing. Modern web testing needs to be capable so that it can actually automate all possible capabilities of the modern web platform. Modern web testing also needs to be reliable, which means it needs to be able to reliably automate user interactions with highly rich and responsive web applications. And finally, modern web testing needs to be fast, which means it needs to be able to speed up our shipping and release processes instead of slowing them down.
3. Capabilities of Playright for Modern Web Testing
Any testing solution for the modern web needs to cover capable, reliable, and fast. Playright is focused on capable automation without trade-offs, reliable automation without flakiness, and fast execution. You can use Playright to automate browser interactions, launch instances of Chromium, Firefox, and WebKit, and test across all platforms. Playright allows for running browsers in headless and head-full modes, providing flexibility for testing modern web applications.
Any testing solution for the modern web needs to cover capable, reliable, and fast, the three pillars of modern web testing. We've actually focused on these three core ideas while building Playright, which means Playright is actually focused on capable automation without any trade-offs. It's also focused on reliable automation without any flakiness or time odds. And finally, Playright enables fast execution, just like unit tests. And so you can actually use end-to-end tests with Playright to be able to actually speed up your test workflows and not slow them down.
Let's see how more of these capabilities work. With Playright, you can actually have browsers running in both headless and head-full modes. This means that I can actually run my tests on CI in headless modes, so that the UI is not visible and my tests actually run faster. But I can also run these tests locally in head-full mode, where I'm actually looking at the UI and seeing what has happened in the browser. This functionality, again, is available across all platforms. When we are testing for our modern web applications today, it's not just about testing them on our desktop environments.
4. Capabilities of Playwright and Reliability
We can emulate mobile environments, use newer web features, intercept network activity, and automate interactions across multiple pages with Playwright. It provides automation without tradeoffs, coverage for modern web platform APIs, and solves the problem of flaky tests by listening to browser events.
We are also building applications for mobile devices. With Playwright, we can actually emulate mobile environments with our cross-browser tests. So when I'm launching an instance of WebKit now, I'm actually specifying that the WebKit page should emulate the iPhone 11 device. This means that my page will have the right viewport just as an iPhone 11, and would enable mobile-specific features like touchscreen events.
Now, talking more about capabilities, we talked about how the web today can detect your current location. When you're launching a browser with Playwright, you can actually use these newer web features. In this particular snippet, I'm actually launching the browser and specifying dark mode to be enabled. I'm also then specifying a particular geolocation for that browser window. I'm specifying a bunch of permissions and local. This way, my browser is actually working as if I have a mock current location. It gives you a quick preview of how Playwright enables newer web features so that you can actually test your applications for those capabilities.
The other important capability that I want to highlight here is that Playwright can also intercept network activity. In this particular snippet, when I launched a browser and created a new page, I set up a mock API so that my API endpoint essentially returns an error code. This gives me the ability to actually test for an error scenario with Playwright's network capabilities. And then finally, another capability to highlight is that Playwright can actually automate interactions across multiple pages. You're not limited to one particular page. In this example, I'm launching an instance of the browser, creating a new page on it, and then opening a new popup. We can then automate that popup with Playwright.
What we just saw was automation without tradeoffs. You were able to automate all modern web browsers, you were able to get headless and head full execution across all platforms. You also saw device emulation for mobile testing, and the best coverage for modern web platform APIs. If you're using any of these APIs, Playwright is really the best tool to be able to author and create your end-to-end tests so that you can actually ensure that all features of your application are covered with your end-to-end tests. That wraps up the capabilities section.
Let's now talk about reliability. As we just talked about earlier, reliability is essentially the problem of being able to automate user interactions on your application without having your tests flake. In a lot of cases, users have to actually author timeouts, which means they need to make their tests sleep for some time so that they can actually reliably automate. This creates a whole bunch of problems because timeouts actually lead to flakiness. Playwright solves these problems in a very unique way. Let's see how. The key thing to note about Playwright is that Playwright can actually listen to browser events.
5. Capturing Events and Enhancing Test Reliability
Playwright can capture events from the browser and understand the application's behavior. By launching a browser instance, creating a new page, and setting up event listeners, you can capture console logs and other events like DOM content loaded. These event listeners enhance the reliability of your tests.
It can actually get events from the browser and actually understand what's happening in your application when it's running inside the browser. In this particular snippet, I'm launching an instance of Firefox, I'm creating a new page, and then I'm setting up a new event listener for that particular page. I'm basically saying whenever my application throws a console log it should show up in my test. Playwright is not limited to console logs. I can also listen for other events, such as the DOM content loaded event. The DOM content loaded event is an important lifecycle event when your application or your web page is actually loaded inside the browser. And I can actually listen for this event as a part of my test.
6. Listening to Events and Enhancing Reliability
Playwright's ability to listen to events gives it the power to wait for precise events, making tests highly reliable. Playwright's page.click method automatically waits for the right conditions, such as element visibility, animation completion, and click event reception. This eliminates the need for custom wait conditions or sleep timers, making tests easier to write and maintain. Playwright also provides APIs for custom wait conditions, ensuring precise and reliable testing. This feature sets Playwright apart from other testing tools and enhances reliability.
I can also listen to network activity. For example, I can check for whether a particular web socket connection has been opened. This ability to listen to events gives Playwright the ability to wait for precise events instead of having to wait for blankets and sleep timers. This makes your tests highly reliable.
Playwright uses this functionality internally on every interaction. In this snippet, I'm launching an instance of Firefox and then I'm going to github.com and then I'm clicking on a button. Notice that I did not have to use any sleep timers. This is because the page.click in Playwright is automatically waiting for the right conditions. It's waiting for the element to be visible, it's waiting for the element to stop animating, it's waiting for the element to receive click events. All of this is internally done for you so that your user interactions can reliably go to your applications and test your applications. This makes your tests significantly easier to write and maintain over time, because you don't have to worry about custom wait conditions or sleep timers. In a lot of cases, these defaults just work out of the box, but in case you're looking for more control, Playwright also provides you APIs to actually offer your own custom wait for conditions. Now, these wait for conditions, again use the same entire event logic, which means that you don't have to worry about guessing the amount of time this would take. You can actually write that precise condition and make sure that your tests actually wait for it. Now, this essentially shows how Playwright is more reliable. It essentially means that you don't need to actually guess on how long something would take. You can actually use events to verify that application state. And this is, you know, wrapped up automatically for all interactions inside Playwright, which means everything that you do in Playwright has built-in auto waits and retries. And then finally, you know, if your application requires custom logic, you have powerful wait for APIs that you can use and actually author your wait for conditions. This feature differentiates Playwright from other testing tools and actually improves reliability.
7. Fast Execution and Browser Contexts
Developers and testers are generally happy with their unit tests as they run reliably and are fast to execute. However, end-to-end tests can be slow due to the dependence on browsers. Playwright introduces a new abstraction called browser contexts to solve this problem. Browser contexts allow for hosting multiple web pages and can be created cheaply, significantly speeding up tests. These contexts are completely isolated from each other, enabling concurrent, isolated testing. Browser contexts also enable emulation, allowing tests to be run across multiple devices.
Now let's talk about fast execution, you know, developers and testers are generally very happy with their unit tests because unit tests run reliably and are fast to execute. However, that's not the case with end-to-end tests because end-to-end tests depend on browsers and browsers can be slow, especially when it comes to launching and closing a browser. If you're running hundreds of tests, being able to sort of run hundreds of browsers is a very expensive operation and that actually drastically makes your tests slower.
To solve this in Playwright, we've actually introduced a new abstraction there. Let's walk through this code snippet to show you how it works. So in this snippet, I'm using Playwright to launch an instance of WebKit, which is the browser engine used inside Apple Safari. Now in this browser instance, I'm actually creating a new context with the browser.newcontext API. Despite the context, I'm then now opening my pages and doing interactions like I was doing earlier. What we've done here is that we've created a new abstraction layer on top of this browser instance called the browser context. This browser context essentially can host multiple web pages inside it. Now, it's very cheap to create browser contexts, which means you can actually spin up multiple contexts without having to worry about the time it takes to launch and close a browser. This significantly speeds up your tests.
Now, there are a few properties of browser contexts that I want to highlight. The first is that browser contexts are actually completely isolated from each other, which means that you can actually create multiple browser contexts, run isolated tests without having to worry that they would affect each other. In this particular code snippet, I'm actually launching an instance of Firefox, and then I'm creating two contexts in it. The first context is an empty context, and the second context is context with authentication. In the second context, I'm adding particular cookies. Now, this essentially means I can actually run my test in two different scenarios. The first is no authentication, empty state, which is a great way to test your marketing websites. The second context is context with authentication, which means you can actually test for workflows that require authentication. Cookies in the second context will not affect the first context. This basically ensures that if we have hundreds of tests, they can run in concurrent, isolated environments through browser contexts without having to spend time to launch and close browsers.
The other interesting thing that browser contexts enable is emulation. We looked at how you can use Playwright to automate device testing for iPhone or iPad devices. In this particular example, I'm actually launching an instance of WebKit, and then I'm actually specifying the two devices I want to test, the iPhone 11 and the iPad Pro 11. When I'm creating this browser instance, I can then create browser contexts on it. I'm basically looping over the devices to test and essentially creating a new context for every device that I want to test. Once I have that browser context, I can then open multiple pages inside it and navigate to the pages that I want to. This essentially means that we had one browser instance but we were able to actually reuse that browser instance in form of browser contexts to be able to run multiple tests across multiple devices. Browser contexts, therefore, not only make your tests faster, they also give you a lot more capability to be able to run emulation scenarios, for example.
8. Playwright: Faster, Capable, and Reliable
Playwright makes end-to-end tests faster by using browser contexts for parallelized execution. It enables capable, reliable, and fast automation, supports multiple languages and frameworks, and can be used in any CI-CD environment. Learn more at playwright.dev, GitHub, and join our Slack community.
This is essentially how Playwright is making your end-to-end tests a lot faster, right? Despite browsers being slow, we're able to create a new abstraction layer called browser contexts which are much faster. Browser context parallelizes execution with a single instance, which means you can just launch one browser but then actually create parallelized execution environments. Each of them are isolated from the other, and you can have multiple pages being opened inside one browser context. This makes your tests significantly faster and gives you more capabilities like we just talked about.
Capable, reliable, and fast. These are essentially the three core ideas behind Playwright. We've talked about how Playwright enables capable automation without trade-offs, how it enables reliable automation without having to worry about flakiness or time outs, and how it enables fast execution, just like your unit tests.
And finally, you know, to learn more about Playwright, just wrapping it up with a few interesting links. The first is our documentation website at playwright.dev. You can also find us on GitHub in the Microsoft org under the Playwright repo name. And finally, you can join our Slack to actually interact with other users who are using Playwright for their end-to-end tests and also reach out to the Playwright team. We are all on Slack. Thank you so much for attending this talk.
9. Companies Using Playwright and Its Capabilities
10. Cross-Browser Support and Roadmap
Other languages are possible. Cross-browser support on Playwright is better, including testing for WebKit. Playwright's roadmap focuses on improving the authoring and debugging experience. The integrated test runner simplifies getting started, and upcoming releases will include a debugging UI. Playwright values user feedback and encourages participation in shaping the roadmap through GitHub and Slack.
So, other languages are also possible. Even cross-browser support on Playwright is better in the sense that you can actually test for WebKit, which is the browser engine used inside Apple Safari. And notably, the Playwright support for WebKit is cross-platform. means you can actually use WebKit testing on Mac OS or on Linux or Windows. So, if you have a mobile web application, which is used by users on Safari, and you have a Linux CI CD agent, you can actually use Playwright to run your automated tests for WebKit essentially. So, better browser support, better language bindings, and capability, automation really without any tradeoffs is really our pitch for Cypress.
Super nice that you can also run this WebKit on a Windows machine. I think that really helps if you have a big team when people are using Mac OS and Linux and Windows, of course. Super powerful, I guess.
Next question is from Danny. Can you share something about your roadmap for this in the upcoming year? Yeah, for sure. I think what's been amazing actually is we've seen lots of amazing traction over the last one year. And with any sort of new tool, there's tons of user feedback, tons of GitHub issues or Slack community messages or just engaged users getting onto conversations with us. And that is essentially what's helping us shape our roadmap for this year. Our current focus is on making the getting started experience of authoring your tests even easier. And sort of debugging your tests easier. So one of the things that we released in preview last year was a test runner, an integrated test runner for Playwright. Which means you don't actually necessarily have to use something like Mocha or Jest and figure that integration for yourself. With the integrated test runner, you'll actually get a full, complete end-to-end testing tool which would simplify the experience of getting started significantly. Similarly, maintaining your tests over time is obviously the hardest part when it comes to end-to-end testing and better debugging is something that a lot of our users have asked for and something that we are... We always wanted to build that, but this is a priority for this quarter. We've been doing monthly releases. Playwright 1.8 was live last week. Playwright 1.9 goes live in February and actually has an amazing debugging UI which will actually help you maintain your tests over time. So, lots of authoring and debugging improvements, really, which will be live over the next quarter or so. And that's really our priority. So, that's our roadmap for Playwright. And people can just influence it by joining your Slack server, I understand. So, Danny, if you want to influence it, you can do it. Yeah, you know, I think that's the beauty of- Oh, sorry. Go ahead. Yeah, I was just saying, you know, that's the beauty of open source software, right? I mean, GitHub issues or Slack, you know, our product development and roadmap is fully transparent and we are always seeking feedback. All right. So, Danny, if you want to add something to the roadmap, you can just do it yourself. For the rest of the questions, Arjun is now going to go to his speaker room. And if you have want to talk to him more about Playwright, now is the time. Arjun, I want to thank you a lot for joining us here. And I hope to see you again soon at the next big event.