1. Introduction to Browser Automation
I'm Michael Hablich, a product manager on the Chrome team, working on reducing friction of testing and debugging web applications. Today, I'll talk about browser automation techniques and why we're adding a fourth one, web driver. Quality assurance and testing activities take up a big chunk of the software development cost, and test automation is a very good way to reduce the continuous costs of testing. Browser automation automates user interactions and pretends to be a user, with typical use cases including test automation, web scraping, and rendering part of pages like ads. Let's take a short tour of the history of browser automation, from the native APIs in the 90s to the complexities of Java applets and Flash in the 2000s.
Hi folks. I'm Michael Hablich, a product manager on the Chrome team. There, I'm working on reducing friction of testing and debugging web applications. I have the honor today to talk about browser automation techniques and why we're adding a fourth one, web driver by the way.
I spent around 20 years working in Tech already. A big chunk of this is building test automation solutions for enterprises. One can say I had a lot of fun automating browsers, .NET applications, and more niche technologies like Power Builder.
So, why I'm here? Well, the Chrome team periodically reviews the satisfaction of web developers and, surprise, testing, in particular, across browsers, is a top pain point for web developers. Quality assurance and testing activities take up a very big chunk of the software development cost, and you can't simply cut them away. QA is necessary because either your testing applications or your users are filled, and the latter has some risk attached to it. And test automation is a very good way to reduce the continuous costs of testing.
2. Browser Automation Techniques
Developers started to build very rich and interactive experiences on the web. YouTube and Google Map are some very good, early examples of this. With smartphones coming into the picture, needs for test automation increased because suddenly there was a requirement for cross-browser and cross-device compatibility. Selenium and the WebDriver project were born to solve the test automation challenges.
3. Automating Test Cases with Browser Drivers
Let's assume you're a web developer or a tester who wants to automate a test case. Your automation tools translate your scripts into HTTP and communicate with the browser drivers through web driver commands. The browser drivers then communicate with the browser via internal browser-specific protocols. Let's take a look at an example of a script in WebDriver I.O. Each action is translated into HTTP requests. The browser drivers handle the requests and send back the response over the same HTTP connection. However, waiting for elements to appear on the screen can be tricky, as the browser drivers are not able to notify the automation libraries. The libraries need to constantly send requests to check for the status.
Let's assume you're a web developer or a tester and that wants to automate a test case, the most common use case for browser automation. You pick a test automation tool and write tests in it. Then run these tests as part of your CI. Behind the scenes, your automation tools will translate your script and run them in the browser through some sort of protocol, web driver or CDP, for example.
Here you can see that there's an entity added between automation tools and browsers, the browser drivers. Typically these need to be installed separately to automate a browser via web driver. Your automation tools translate your scripts into HTTP and communicate with the browser drivers through web driver commands. Browser drivers then communicate with the browser via internal browser-specific protocols. Since WebDriver Classic is a web standard, it is well supported across all major browser vendors. For every new release, these browsers will update and publish a new version of the driver.
Let's take a look at some actual code. Chaseline, Chrome Tooling's excellent developer advocate, created awesome demos. Now I'm going to border them in the coming slides. So thank you, Chaseline. Let's take a look at an example here. Let's say you have a script to navigate to a page and click on a coffee to add it to your shopping cart. This is how the script looks like in WebDriver I.O. Each action will be translated into HTTP requests, for example. What happens when you set the window size is your automation tools send an HTTP post request to change the window. Here's a demo of setting a viewport size in the Safari browser. In our example, these are the three HTTP commands that happen behind the scenes. Most of the time, the browser driver handles the requests and sends back the response over the same HTTP connection. However, the second step is a little bit tricky. A lot of time you need to wait until the element is shown on the screen. In our case, the coffee is loaded from a network request. We need to wait for that before we can find and click it. Due to the nature of HTTP, the browser drivers are not able to notify the automation libraries when the coffee is ready. The libraries need to constantly send a request. Is the espresso there? Is it now? Is the element and so on. And the browser driver is going to check for the status.
4. Automation Protocols and Chrome DevTools
The web driver is lower compared to other protocols because each classic command requires an HTTP handshake. WebDriver Classic has the best cross-browser support but has limitations on supporting some low-level controls. The Chrome DevTools protocol enables debugging and automation of websites. Puppeteer uses CDP under the hood to communicate directly with Chromium-based browsers. CDP supports more low-level controls and features like intercepting network requests and simulating device mode.
From the example above, we know that the web driver is lower compared to other protocols because each classic command requires an HTTP handshake. Obviously, this is just a simplification and there are techniques to mask and mitigate this to a certain extent.
In summary, WebDriver Classic has the best cross-browser support. However, it is lower and has limitations on supporting some low-level controls, which we will look into later.
Next, let's take a look at the Chrome DevTools protocol. From the name, you can guess that this protocol is designed to enable Chrome DevTools to debug webpages. It turns out that a lot of the features that are needed for debugging a website can also be used to automate a website.
For example, it doesn't really matter if we want to get the inner text of an HTML element in order to display it in Chrome DevTools or to verify the inner text in one of your tests. Since the protocol is fast and powerful, Puppeteer uses CDP under the hood for automation purposes. Different from WebDriver, CDP communicates directly with Chromium-based browsers. There is no browser driver needed, as they are essentially already included. Automation tools issue commands through CDP, and these commands are sent to the browser via WebSockets.
Here's a quick refresher on our demo. We're going to navigate to a page, and then click on the Espresso entry to add it to our cart. This is how Puppy The Script looks like for our coffee. It looks very similar to the previous example with WebDriver I.O. Each action will be translated into CDP commands. For example, what happens when you navigate to a page is the following. Roughly these commands here. First, we navigate to a page, then find a coffee, and third, add it to our cart. Here, for example, you can see issuing these CDP commands directly in DevTools with a protocol monitor. In case you want to try it out yourself, you will need to enable the protocol monitor in DevTools settings, but whatever.
Going back to Puppeteer and how it uses CDP to automate the browser. CDP actually uses WebSocket, therefore the communications are bidirectional by default. Once the browser completes the request, it sends the updates back to the tools automatically. No polling is needed to wait for the coffee, so there is no artificial delay. Also, since CDP is designed to cover all debugging needs, it supports more low-level controls compared to WebDriver. It supports features like intercept network requests, simulate device mode and geolocation, as well as getting console messages and so on. Back in time when the WebDriver protocol was developed, there was no need for low-level control,. However, times have changed and testing now requires more and more fine-grained actions.
5. WebDriver Bidirection Project
Although CDP is fast and with low-level control, it only works in Chromium-based browsers, and it's also not a standard. WebDriver is relatively slow and not low-level enough. The WebDriver Bidirection project aims to combine the good parts of both protocols, offering bi-directional messaging, low-level controls, cross-browser support, and standardization. It complements WebDriver and reduces the need for automation libraries to directly use CDP. However, the WebDriver protocol is still a work in progress, with specifications and implementations being finalized by browser vendors and test automation library vendors.
Although CDP is fast and with low-level control, it only works in Chromium-based browsers, and it's also not a standard. So, what we can see is both browser automation protocols have their drawbacks. CDP is not a standard and browser-specific. WebDriver is relatively slow and not low-level enough. But both protocols also offer unique benefits.
CDP is fast and bi-directional. It provides low-level control over the browser. WebDriver is built for testing and the standard and it's supported cross-browser. So, what if we only take the good parts from both protocols? That is what the WebDriver Bidirection project is about. It is a new standard for browser automation, combining the good parts of WebDriver and CDP. It has bi-directional messaging, low-level controls, cross-browser support and standardization.
It's built for testing and not for debugging. This diagram looks very similar to what you have seen already for CDP and WebDriver. Well, the reason is because WebDriver Bidirection works like a combination of both of course. WebDriver Bidirection commands are sent via WebSocket connections. The receiver can be a classic driver like Chrome driver or the browser directly. A few things are important to note here though. WebDriver Bidirection should complement WebDriver. Bidirection commands and classic commands can be run next to each other. There's no need for a big migration. I will talk about this in a few minutes in a little bit more detail. Also, the need for automation libraries to directly use CDP should drastically decrease. In fact, we are working on making Puppeteer also use Bidi instead of CDP directly under the hood. As mentioned before, WebDriver Bidi combines the benefits of CDP and WebDriver. It's important to note that this new standard is a collaboration of various browser vendors, test frameworks, and test infrastructure providers.
Now, the bad news. There is always bad news, right? So, the WebDriver protocol is still a work in progress. Browser vendors and test automation library vendors are still working on finalizing the specifications and the implementations. If you are interested in more detail, check out the handy QR code, which will open a dashboard showing browser support for WebDriver Bidi. Enough of the bad news, though.
6. WebDriver Bidi and Console Monitoring
WebDriver Bidi is still in progress, but parts of it are already being shipped incrementally. Automation libraries like Selenium, WebDriver IO, and Puppeteer have initial Bidi support. WebDriver Bidi allows monitoring of console messages, which can be useful for testing and error detection. A demo shows how to set up the browser, enable the web socket URL, and monitor log messages with ByteEye. This functionality works in both Firefox and Puppeteer. WebDriver ByteEye combines stable automation and is a good choice for browser automation. You can try it out today if your browser and test automation library support it, and provide feedback.
Let's talk about the good news. They are more important anyway. I just told you WebDriver Bidi is still in progress. That's true. We are shipping in parts of WebDriver Bidi incrementally, though, which means you can actually already start using it today. Automation libraries like Selenium, WebDriver IO or Puppeteer have landed initial Bidi support.
What does this mean in practice for you as a web developer? How can you benefit from this today? Well, if you look at the link I provided before, where one can track cross-browser support of WebDriver Bidi. It turns out that WebDriver Bidi has the capability to monitor console messages. That capability is part of the Red Circle log entry there. How is this going to be useful? Let's show it again with a demo provided again by Jesslyn. Let's go back to a previous example where we are ordering a coffee. So it's common that we call some additional analytics API when a user does some action like adding coffee to the cart. And we want to make sure that these APIs are called to the test. And we want to monitor for error surface to the console. So in order to find out if something is going wrong.
This demo here is going to use Web Drive AO as an example. First, we need to set up the browser and enable the web socket URL in the settings. In this case, I use Chrome here, but it works in Firefox as well. So after the setup, we can start to monitor the log messages with ByteEye. There are two parts of the code that make it happen. First, you need to subscribe to the log event in the session. And then you can listen to the event and handle it. In this case, we would just lock the data, but it can do a lot more with it of course. So this is a demo, which are running Firefox. Observe the output. The console message and the error from the webpage are captured successfully and processed. This also works in Puppeteer, which is also starting to use WebDriver, ByteEye under the hood. Here you can see the same thing done in Puppeteer. Launch Puppeteer with the WebDriver ByteEye protocol. Then you're set to monitor the event with ByteEye instead of CDP.
So let's wrap it up. Browser automation is hard. There's not the one way to automate the browser. WebDriver ByteEye combines the benefits of the most stable automation which in my completely unbiased view, make it clearly a good choice. You can try out the first slices of WebDriver ByteEye already today, if your browser and the test automation library supports this, of course, and if you're doing that, don't forget to give us feedback.