JS Nation

JSNation 2023

A journey through overwhelming ways to automate browsers. Join Michael on a journey to see what happens behind the scenes of "await page.goto('https://example.com');" et. al. See what pros and cons each of the three ways of browser automation have. Understand why we are adding a fourth - WebDriver BiDi.

Three Ways to Automate Your Browser, and Why We Are Adding a Fourth: WebDriver BiDi

Hi folks, I'm Michael Hablich, a product manager on the Chrome team. There I'm working on reducing friction of testing and debugging web applications. I have the honor today to talk about browser automation techniques and why we're adding a fourth one, WebDriver by the way. I spent around 20 years working in tech already. A big chunk of this is building test automation solutions for enterprises. One can say I had a lot of fun automating browsers,.NET applications, more niche technologies like Power Builder. So why I'm here? Well, the Chrome team periodically reviews the satisfaction of web developers and surprise, testing in particular across browsers is a top pain point for web developers. Quality assurance and testing activities take up a very big chunk of the software development cost and you can't simply cut them away. QA is necessary because either you are testing applications or your users are filled and the latter has some risk attached to it. And test automation is a very good way to reduce the continuous costs of testing. So let's first define a bit what browser automation is about and briefly skim how it works. Simplified browser automation automates user interactions and pretends to the browser to be a user. Often such interactions are stored as source code as seen on the left side. These interactions are then replayed as you can see on the right side. Typical use cases of browser automation technologies are test automation, web scraping or rendering parts of pages like ads. Today I'm focusing on the first test automation. The previous slide showed the current state of browser automation. Tests defined in JSON and JavaScript, fast and stable automation and so on. Before we ended up in such a cozy place, a lot of history has happened. Let's take a short tour. The web was born in the 90s. People started using browsers in a limited set of big screen devices. Testing in these decades is mostly done against data content. Apps like Netscape Navigator or Internet Explorer were shipped. Browser automation at that time was done via native APIs. For example, I can still remember using Visual Basic 6 to automate Internet Explorer. In 1996, Java applets and Flash became a thing. They made automating web pages even more complicated because the browser automation APIs provided by the browser vendors did not work for Java applets and Flash containers. Single testing or injecting scripts were the way to go for these technologies. In the 2000s, more browsers were joining the scenes, including Chrome. Developers started to build very rich and interactive experiences on the web. YouTube and Google Map are some very good early examples of this. With smartphones coming into the picture, needs for test automation increased because suddenly there was a requirement for cross-browser and cross-device compatibility. Selenium and the WebDriver project were born to solve the test automation challenges. At that time, it was common to write Selenium tests in Java. In 2009, Node.js brought JavaScript development to the back end. Also it enabled running tests written in JavaScript. More JavaScript frameworks came into the picture. At the same time, Selenium and WebDriver merged into a single Selenium WebDriver project. With the growing popularity, the project became a W3C standard in 2018. And we call it WebDriver Classic. With more developers building rich applications in JavaScript, these developers also wanted to perform test automation in JavaScript as well. Multiple web-based JavaScript testing libraries are introduced to address the needs. And not all of them use WebDriver as the underlying automation technology. They're using different techniques to automate the browser, which we're going to talk about today. We will cover the WebDriver protocol, supported by solutions like Selenium, Nightwatch.js, or WebDriver IO. The Chrome DevTools protocol, CDP in short, powering Puppeteer, Chrome's own automation library, and Playwright. And web APIs plus browser extensions, leveraged by TestCafe or Cypress, for example. Let's start and take a step back and talk about how tools automate browsers. I mentioned three major ways to automate a browser. Well, the following two major categories too. Let's intensify the complexity a bit, because we have high level, which executes JavaScript injected into the browser, and low level, which executes remote commands. For example, Cypress utilizes browser extensions and Node.js to execute tests directly in the browser. To gain greater control of the browser, like opening multiple tabs and testing for party iframes, we need to go deeper and execute remote commands. With other techniques, let's call them simply protocols. The two common protocols are WebDriver and Chrome DevTools protocol, CDP in short. We will explore all of this together shortly, no worries. I'm going to start with the approach to use web APIs and browser extensions to build your own automation layer. Essentially, the solutions leverage a melange of web APIs, JS injection, browser extensions, proxies, et cetera, to build their very own automation layer. Going into detail here would burst the talk, the size of the talk. So I'm going to stop here and segue over to WebDriver, the automation technique built upon standard. It's one of the low level protocols, so let's take a brief look how they work in principle. Let's assume you're a web developer or a tester, and that wants to automate a test case, the most common use case for browser automation. You pick a test automation tool and write tests in it, then run these tests as part of your CI. Behind the scenes, your automation tools will translate your script and run them in the browser through some sort of protocol, WebDriver or CDP, for example. Here you can see that there's an entity added between automation tools and browsers, the browser drivers. Typically, these need to be installed separately to automate a browser via WebDriver. Your automation tools translate your scripts into HTTP and communicate with the browser drivers through WebDriver commands. Browser drivers then communicate with the browser via internal browser specific protocols. Since WebDriver Classic is a web standard, it is well supported across all major browser vendors. For every new release, these browsers will update and publish a new version of the driver. Let's take a look at some actual code. Chesslane, Chrome Tooling's excellent developer advocate, created awesome demos. I'm going to borrow them in the coming slides, so thank you, Chesslane. Let's take a look at an example here. Let's say you have a script to navigate to a page and click on a coffee to add it to your shopping cart. This is how the script looks like in WebDriver I.O. Each action will be translated into HTTP requests. For example, what happens when you set the window size is your automation tools send an HTTP POST request to change the window. Here's a demo of setting a viewport size in a Safari browser. In our example, these are the three HTTP commands that happen behind the scenes. Most of the time, the browser driver handles the requests and sends back the response over the same HTTP connection. However, the second step is a little bit tricky. A lot of time you need to wait until the element is shown on the screen. In our case, the coffee is loaded from a network request. We need to wait for that before we can find and click it. Due to the nature of HTTP, the browser drivers are not able to notify the automation libraries when the coffee is ready. The libraries need to constantly send a request. Is the espresso there? Is it now? Is the element there? And so on. And the browser driver is going to check for the status. From example above, we know that the WebDriver is slower compared to other protocols because each classic command requires an HTTP handshake. Obviously, this is just a simplification and there are techniques to mask and mitigate this to a certain extent. In summary, WebDriver Classic has the best cross-browser support. However, it is slower and has limitations on supporting some low-level controls, which we will look into later. Next, let's take a look at the Chrome DevTools protocol. From the name, you can guess that this protocol is designed to enable Chrome DevTools to debug web pages. It turns out that a lot of the features that are needed for debugging a website can also be used to automate a website. For example, it doesn't really matter if you want to get the inner text of an HTML element in order to display it in Chrome DevTools or to verify the inner text in one of your tests. Because the protocol is fast and powerful, Puppeteer uses CDP under the hood for automation purposes. Different from WebDriver, CDP communicates directly with Chromium-based browsers. There is no browser driver needed, as they are essentially already included. Automation tools issue commands through CDP, and these commands are sent to the browser via web sockets. Here's a quick refresher on our demo. We're going to navigate to a page and then click on the Espresso entry to add it to our cart. This is how Puppeteer's script looks like for our coffee. It looks very similar to the previous example with WebDriver IO. Each action will be translated into CDP commands. For example, what happens when you navigate to a page is the following. Roughly these commands here. First we navigate to a page, then find the coffee, and third, add it to our cart. Here for example you can see issuing the CDP commands directly in DevTools via the protocol monitor. In case you want to try it out yourself, you will need to enable the protocol monitor in the DevTools settings. But whatever. Going back to Puppeteer and how it uses CDP to automate the browser. CDP actually uses a web socket. Therefore the communications are bidirectional by default. Once the browser completes the request, it sends the updates back to the tools automatically. No polling is needed to wait for the coffee, so there's no artificial delay. Also since CDP is designed to cover all debugging needs, it supports more low-level controls compared to WebDriver. It supports features like intercept network requests, simulate device mode and geolocation, as well as getting console messages and so on. Back in time when the WebDriver protocol was developed, there was no need for low-level control. However, times have changed and testing now requires more and more fine-grained actions. Although CDP is fast and with low-level control, it only works in Chromium-based browsers and it's also not a standard. So what we can see is both browser automation protocols have their drawbacks. CDP is not a standard and browser-specific. WebDriver is relatively slow and not low-level enough. But both protocols also offer unique benefits. CDP is fast and bidirectional. It provides low-level control over the browser. WebDriver is built for testing and the standard and it's supported cross-browser. So what if we only take the good parts from both protocols? That is what the WebDriver BIDAP project is about. It is a new standard for browser automation, combining the good parts of WebDriver and CDP. It has bidirectional messaging, low-level controls, cross-browser support and standardization. It's built for testing and not for debugging. This diagram looks very similar to what you have seen already for CDP and WebDriver. Well the reason is because WebDriver BIDI works like a combination of both of course. WebDriver BIDI commands are sent via WebSocket connections. The receiver can be a classic driver like a Chrome driver or the browser directly. A few things are important to note here though. WebDriver BIDI should complement WebDriver. BIDAP commands and classic commands can be run next to each other. There is no need for a big migration. I will talk about this in a few minutes in a little bit more detail. Also the need for automation libraries to directly use CDP should drastically decrease. In fact we are working on making Puppeteer also use BIDI instead of CDP directly under the hood. As mentioned before WebDriver BIDI combines the benefits of CDP and WebDriver. It's important to note that this new standard is a collaboration of various browser vendors, test frameworks and test infrastructure providers. Now the bad news. There is always bad news, right? So the WebDriver protocol is still a work in progress. Browser vendors and test automation library vendors are still working on finalizing the specifications and the implementations. If you're interested in more detail check out the handy QR code which will open a dashboard showing browser support for WebDriver BIDI. Enough of the bad news though. Let's talk about the good news. They are more important anyway. I just told you WebDriver BIDI is still in progress. That's true. We are shipping in parts of WebDriver BIDI incrementally though, which means you can actually already start using it today. Some libraries like Selenium, WebDriver IO or Puppeteer have landed initial BIDI support. What does this mean in practice for you as a web developer? How can you benefit from this today? Well if you look at the link I provided before where one can track cross-browser support of WebDriver BIDI, it turns out that WebDriver BIDI has the capability to monitor console messages. That capability is part of the Red Circled Log Entry there. How is this going to be useful? Let's show it again with a demo provided again by Jesselyn. Let's go back to a previous example where we are ordering a coffee. So it's common that we call some additional analytics API when the user does some action like adding coffee to the cart. And we want to make sure that these APIs are called to the test and we want to monitor for error surface to the console. So in order to find out if something is going wrong. This demo here is going to use WebDriver IO as an example. First we need to set up the browser. And enable the WebSocket URL in the settings. In this case I use Chrome here but it works in Firefox as well. So after the setup we can start to monitor the log messages with BIDI. There are two parts of the code that make it happen. First you need to subscribe to the log event in the session and then you can listen to the event and handle it. In this case we will just log the data but it can do a lot more of it of course. So this is a demo which will run in Firefox. Observe the output. The console message and the error from the web page are captured successfully and processed. This also works in Puppeteer which is also starting to use WebDriver BIDI under the hood. Here you can see the same thing done in Puppeteer. Launch Puppeteer with the WebDriver BIDI protocol. Then you're set to monitor the event with BIDI instead of CDP. So let's wrap it up. Browser automation is hard. There's not the one way to automate a browser. WebDriver BIDI combines the benefits of the most stable automation techniques which in my completely unbiased view make it clearly a good choice. You can try out the first slices of WebDriver BIDI already today if your browser and the test automation library supports this of course. And if you're doing that don't forget to give us feedback. And with that I wish you a pleasant day and happy testing.
