The Evolution of Browser Automation

Rate this content

In this session, we’ll take a look at what has happened behind the scenes in browser automation throughout the years and what the future will have in stock for us. We will examine how web testing will develop and what challenges this will bring for conventional frameworks like Selenium or WebdriverIO, as well as new frameworks such as Cypress, Puppeteer and Playwright. Lastly, we will experiment with some new automation capabilities these frameworks provide to test some of the new web features.

34 min
15 Jun, 2021

Video Summary and Transcription

Browser automation has evolved over the years, starting with Selenium and WebDriver. Tools like Cypress, Taskerfee, Puppeteer, and Playwise use different approaches for automation. The new WebDriver protocol will enable sending and receiving thousands of commands and messages simultaneously. New testing types, such as performance and accessibility testing, will continue to emerge. The new WebDriver protocol combines the best of all three approaches and provides opportunities for testing and automating web applications.

Available in Español

1. Introduction to Browser Automation

Short description:

Hello, y'all. Thank you for joining the session. I'm Christian, working at Sauce Labs. Let's talk about browser automation and its misconceptions. Browser automation has evolved over the years, starting with Selenium and WebDriver. Jason and Simon merged the projects to overcome limitations. A working group at the W3C was formed to standardize the process.

Hello, y'all. Thank you for joining the session, and particularly big thanks to the TestJS Summit organizer and speaker committee for inviting me to open the conference. I'm very excited about all the great talks from experts around the world that we will get to see over the next two days. And I'm very happy to see such great events continue to take place despite the difficult global situation that we find ourselves in.

I would love to spend the next 25 minutes to speak a little bit about how browser automation has been involved over the last decade or so, and I hope it gives a little bit more context when you hear about automation tools in the upcoming sessions. But before we start, let me introduce myself. I'm Christian, I'm working in the Open Source Program Office at Sauce Labs. And most people probably know me as the maintainer of Web.IO, which is a project that got me excited about automated testing and browser automation many, many years ago. And maintaining the project really taught me a lot of things about generally how browsers work and how open source and standards are being developed. And those are all topics that I'm fortunate to work on full times these days.

The reason I wanted to give this talk is because I see a lot of misconceptions about how browser automation actually works. It's an interesting challenge, especially for cloud vendors and cloud providers, because as a user there's not much delineation between your automation framework and the automation actually happening in the browser. So for instance, if your click doesn't happen, even though the test script passes without errors or if the script cannot find an element, even though you clearly see that the element is there when you check it yourself, people kind of blame the frameworks first and then at some point the cloud vendor second. While in reality, there are a lot of nuances and processes responsible for making that click happen in the first place, maybe in a VM that is miles and miles away from the machine that actually runs your test. So let's have a look how a click command in the framework actually ends up being a click event in the browser and to do so, I would like to start with a small recap.

Browser automation has been around for more than a decade and there have been quite some interesting developments and influences happening over the years, especially with the web changing from how it was 15 years ago to what it is today. So let's recap what has happened so far and how we got where we are right now. So it started all kind of in 2004, with someone called Jason Huggins, having the need to test an expense tool at ThoughtWorks to make sure it works on IE as well as on Firefox back then. He called that tool Selenium, and it's probably a project that you all know already. A year later, another actor jumps into the scene, claiming to have built a better tool, which was called WebDriver. That guy was Simon Stewart. Both tools over the years gained more and more popularity as browser automation became a thing to test web applications. So at some point, Jason thought it would be a good idea to create a company. That company was called Source Labs. It was apparent that both tools, the WebDriver project and the Selenium project, they were great, but they have their certain limitation in specific areas. Selenium, which was back then running in the browser, had problems with cross-region policies and automation around the browser in general. WebDriver had other limitations when it came around automating certain elements. Jason and Simon merged the project together in joint forces to overcome this limitation and provide the best experience possible at the time. Over the years, these frameworks gained more and more popularity to a point where people had a key idea about how automation works. And so, a working group at the W3C was formed to kind of standardize this process.

2. Evolution of Browser Automation

Short description:

The goal was to ensure consistency across browsers and draft a standard for browser automation. This led to the development of WebDriver IO and Appium. However, the web landscape changed with dynamic JavaScript-heavy applications, decoupled front-end and back-end, and the emergence of new web APIs. Tools like Cypress filled the gaps, while the recommended standard fell short. A new protocol was developed to address modern web app requirements. Conventional tools like Selenium use the WebDriver protocol, while non-standard tools offer their own advantages and limitations. Projects like Cypress, Taskerfee, Puppeteer, and Playwise use different approaches for automation.

The goal here was really to make sure that a click in, for instance, Chrome was the same as a click in Firefox. And so, the people there started an effort to draft a standard with the requirements in mind that people had at this point in time when it comes to browse automation. This created a lot of confidence and traction in the ecosystem, where a lot of new projects started to flourish and started to be created. We see the WebDriver IO release in 2011, and we see other projects like Appium that bring the same principle into the mobile space.

What then happened was quite interesting. The web kind of changes a lot, and also the way how web applications are built. What happened before was kind of a static server that was delivering static web size, has now become a more and more dynamic JavaScript-heavy web application that uses frameworks like React, Vue, Angular, or Swelt. That drastically has changed a lot of requirements that people had when they test applications. Suddenly, front-end and back-end became more decoupled, and people really wanted to start focusing, testing only the front-end application, rather than deploying the whole stack. With the continuous development of more web APIs that became available in the browser, people had more and more use cases to test. A lot of these use cases were not really in focus when WebDriver or SitAnywhere developed. Luckily, during those times, we had companies like Cypress who stepped in and filled the needs for developers in a really extraordinary way. They tried to close the gaps, as well as a lot of other tools that started to pop up in the ecosystem.

During all these developments, the standard that was supposed to solve these problems was finalized and became a so-called recommended standard. However, while it allowed you to run automation across all browsers, its original design was already behind, and it was clear that it wouldn't solve the problems that developers have building modern web applications today. So almost at the same time, a new effort was started to develop a new protocol with experiences and learnings that made creating the first one and new requirements that developers have building modern web apps today. So if we look into the ecosystem, we can pretty much group tools into kind of two buckets. We have, on the one side, the more conventional tools like Selenium or, and we have, on the other side, the so-called non-standard tools. And both groups have some interesting characteristics. Starting with the conventional ones, they are, as you might expect, all using the WebDriver protocol and therefore allow you to truly do cross-browser automation. Everyone, every command you can run in WebDriver is tested in every browser like any other standard that you have in the web. However, given the way some front-end frameworks are built, it can still create some incompatibilities when testing web apps. So as a design of this protocol, which was originally do everything a user would be able to do, it's not very suited for developers that like to introspect all areas of the application. These tools aren't really that popular among devs and more used by QA folks. However, many of them are open governed, open-source projects with a long history and a large community with ReptiveIO that is for instance part of the OpenJS Foundation alongside NodeJS, Mocha, and WebPack and we have Selenium which is the project of the Software Freedom Conservancy. Now on the other side we have the I call them non-standard tools which all have their own ways to automate the browser and their own set of advantages and disadvantages over each other. These custom approaches usually are based on some sort of JavaScript emulation or through the use of browser APIs that makes them however all limited to a certain browser and provides them however with the capabilities that you would not have with WebDriver and therefore is much more interesting for developers that like to introspect web apps they like to introspect the network and the dome things like that. What is interesting that all these projects are paid by companies and multiple people are working on these projects full-time Looking at all these projects together we see that we have tools like Cypress and Taskerfee that taking the approach of using web APIs for automation, we have Puppeteer and Playwise that rely on native browser APIs and lastly we have Selenium, and many other tools that rely on the WebDriver protocol. What's worth pointing out here that some tools like Cypress or actually use a mixture of two approaches. For some automation capabilities Cypress needs to use browser APIs for example to take screenshots of the browser.

3. Automating Browsers with Javascript and Web APIs

Short description: uses browser APIs for performance testing and integration into Google Lighthouse. There are three common approaches: Javascript, browser APIs, and browser driver. Javascript and web APIs are the first generation of browser automation, dating back to Selenium. It offers full control over the execution environment but has limitations. Workarounds are required for certain tasks like taking screenshots.

On the other side you have that uses browser APIs for its performance testing features or integration into Google Lighthouse. Let's have a little bit a little closer look to see how you can automate browsers these days in general. There are as I mentioned before generally three common approaches. One through Javascript, the other through browser APIs and the last through a browser driver. I would order using Javascript and web APIs as automation approach as kind of the first generation of browser automation. Even though more novel tools like Cypress or Taskerface are using this approach it actually has been around since the creation of Selenium, which was like Cypress, a test runner within the browser. In fact for some automation commands, like finding out if an element is visible, Selenium still uses a Javascript that it injects into the page. This is because it is almost impossible to define a visible state in the context of a web standard like WebDriver. The advantage of this approach is that you have full control about the execution environment of your application, and it allows you to run the commands fast and reliable. However, you can only do as much as Javascript and particularly the web APIs allow you to do, which is a lot, but you need some workarounds for certain situations like for instance taking the screenshots.

4. Approaches in Javascript Automation

Short description:

When using Javascript automation as an engine, there are two common approaches: Cypress and TestCafe. Cypress runs the application in an iframe and uses a test runner in the parent frame to access the execution context. TestCafe uses a proxy to inject their own JavaScript into the HTML page. Browser APIs, the second generation of browser automation, have evolved to allow better introspection of browser behavior. However, they are not well-documented and vary across browsers. The WebCounter protocol combines the previous approaches and is a web standard developed by all browser vendors.

If we look how it works when using Javascript automation as an engine, there are usually two common approaches that you can take to go with that approach. One used by Cypress where you run the application on a test in an iframe and have a test runner in the parent frame accessing the execution context. The browser is pretty much a test runner that renders the application on the test in an iframe.

The test runner then can exchange messages or access the iframe with JavaScript directly. TestCafe approaches this a bit differently. They use a proxy to proxy all the requests that come from the page and inject therefore their own JavaScripts into the HTML page. Again, the general limitation to this approach is that you're caught in the JavaScript sandbox and don't have access outside of it. So the workarounds that tools like Cypress or TestCafe use are through browser APIs.

Browser APIs are I would say, the second generation of browser automation because it was originally used in the WebDriver project to automate browsers. Back then it was common to abuse the browser extensions to trigger certain events in the browser. And other browsers like the Internet Explorer were automated by a native common interface that Microsoft provided in Windows operation systems back then. Today, the browser vendors have evolved and browsers are much more capable to introspect what's happening in the browser. And we see a lot of those capabilities are used not only in automation space, but also for debugging browser through tools like Chrome DevTools.

Again, the problem with these browser APIs are that they work very differently and aren't really well documented outside from browser teams. And they actually change a lot from one version to another. Think about it as you have three browser friends that offer your help, but all speak a different language with one friend being Zafari, who doesn't really like to listen to you and always wears headphones. To get all browser friends one task, you would need to understand and speak all the languages, which is why tools like Puppeteer or frameworks that are based on Puppeteer usually can only automate one type of browser. In this case, Chromium-based one. Luckily we see that browser vendors start to join forces to speak a more similar language, which is why we see, for instance, Puppeteer support and Firefox nightly these days. And we have on the other side, former browser teams implementing a custom adjustments to browser engines to allow you to do the same type of communication between all of these browsers.

Lastly, we have the WebCounter protocol, which I would argue be the third generation as it combines really the previous two approaches to ensure a consistent automation experience across all browsers. It's a web standard, which means it's an active lead developed protocol by all browser vendors. So if we meet as a working group, we have people from Microsoft, from Google, from Apple at one table discussing about that standard. All changes to that protocol are tested, like all web standards are. So if we add a command, that command is tested in all browsers and continuously run to check that we are reliable across browsers. But as I mentioned before many times, the original design of that protocol has failed the requirements of testing modern web apps today. The way how the web product protocol works is that you have for every browser is some sort of driver that can translate well-defined commands like a click on an element into an automation command that the browser understands. The protocol is often and correctly referred as being very slow and outdated. One of the reasons is, for instance, that this translation step of the driver requires an extra request. So you can think about with the web product protocol as if you are as a developer or a test author, you're kind of like managing a browser factory.

5. New Standard for Browser Automation

Short description:

Due to COVID, you have to work from home without direct contact with the factory. An assistant helps you make commands, but the limitations of this approach are problematic for testing modern web apps. The W3 working group is developing an updated standard for browser automation, enabling multiple commands to multiple actors and providing introspection into network, dome, and console. Backwards compatibility remains important.

But of course, due to COVID, you have to work from home. And in this case, your home does not have any telephone or internet, so you cannot contact your factory directly. You need to have an assistance to make commands in that factory. So the assistant comes to you, asks you for a command, then the assistant goes to the factory, executes the command, and at some point comes back with the result. This sometimes can take up to 200 milliseconds, if not more, particularly if you run in the cloud. On a busy day, you sometimes run over thousands of commands in a single test. This obviously is not really an effective way to run a factory. While a single transaction is slow, this approach, however, has the advantage to be really scalable. We have customers that run efficient test tubes, up to a thousand tests in parallel, that can tremendously reduce the execution time of a whole test tube, while a single test can still be slow.

Another advantage is that you can automate not only factories with browsers, but literally all kinds of factories, being it maybe a mobile factory or an IoT factory. If you run, let's say, a browser factory, you have an assistant that understands the inner workings of a browser and can help you to make that command relevant to you. If you have a mobile factory, you have an assistant that understands iOS or Android and can execute your commands in there. So, this idea of having this translator for executing a trivial command, like click on a button in a complex user agent, works quite well. As I mentioned before, your assistant here really is able to do all kinds of commands if it understands the factory. But the limitation is that it always has to go to the factory, execute the command and come back. These limitations are very problematic for testing modern web apps, where you really need to introspect the whole factory itself.

So luckily, the W3 working group has been kicked off a new effort to develop an updated standard that would overcome the problems that we see in automating browsers today. Based on this little thought experience that we just did with this system, this new world of browser automation would look like this. You would be some sort of commander that would oversee not only a single factory but all other domains of your ecosystem. Instead of telling one command at a time, you can give multiple commands to multiple actors at the same time while listening to different inputs from them. It makes you some sort of automation god, so to speak. It comes with quite a lot of interesting features. For one, it tries to address the capabilities by developers today who want to have introspection into network, dome and console type of things. It will likely come with primitives to modify network data without having to use a proxy or it will allow you to introspect the dome to handle elements flawlessly. In addition to that, it will provide you enough access so that implementers can be very creative when building test frameworks. For example, there's a bootstrap script plan that allows you to execute that kind of script before your app is loaded into the execution context. And so that allows you to monkey patch API or install your own automation extensions. But it also gives you access to some extent, it also gives you access to browse APIs to control the browser using native capabilities. And so you can finally talk to your Safari friend. But like all the web standards that we have right now, backwards compatibility is important.

6. The Future of Browser Automation

Short description:

The future of browser automation will bring significant changes to how tests are run. The new WebDriver protocol will enable sending and receiving thousands of commands and messages simultaneously, posing challenges for browser and cloud vendors. Execution models will shift, with tests being moved closer to the browser. Functional testing alone will no longer be the sole measure of application quality. Vendors will be involved throughout the software development life-cycle, capturing information from various sources. New testing types, such as performance and accessibility testing, will continue to emerge. While the future remains uncertain, the WebDriver protocol and frameworks like Cypress and Playwright offer exciting developments. Standards are crucial for improving web quality.

So you will be able to continue to test legacy browsers and browsers across just one version. That said, this protocol is in the making. And like all web standards, feedback is important. So if you automate a browser, which I think all of you do to some extent, come by and provide feedback. This is really important so that the new design of the protocol actually covers the use cases and requirements for everyone.

Another interesting fact here is that with this effort, it will change quite a lot about how tests are being run in the future. Because instead of having one command sent and getting one response back, you will be able to send thousands of commands at a time and receive thousands of messages at a time. So the amount of data exchange will increase a lot, which is particularly interesting and problematic for browser vendors like Salesforce, where the current business model is pretty much like you send a browser command through the internet to a VM in the cloud and get a response back. With the new WebDriver protocol, this probably won't work anymore. If you would run or would try to run this at the same scale with the same amount of browser, you could technically de-DOS your own CI-CDA system. So, those browser vendors or cloud vendors, they need to find a solution to overcome this problem. And there have been some interesting developments in this market. For one, we will likely see a change in the execution model. So we won't send a single command through the internet anymore, because your test lives on the different side of the world than the browser. These commands or components will be moved together. And so the browser vendor will likely either ask you to upload the test to the cloud or provide you a way to run the test with the browser in your own infrastructure.

Another trend I've seen is that functional testing alone is not the only way how we will measure the quality of your application anymore. Vendors will be more apparent throughout the whole software development life-cycle of the application and will accompany you through development, testing, and release process. So information will be captured through all the sides of the spectrum and used to indicate what is actually wrong in your application. And lastly, with the increase of all these quality signals, we will also see an increase in new testing types. So over the last years, we already have seen development in that space. For instance, there have been development and performance testing with WebDiver. This year, we will likely see more accessibility testing tools and there will be more different kind of testing types like that. So there will be a lot of interesting developments and while the protocol will make us a lot of work as a browser vendor, it also provides tremendous amount of opportunities to build the tooling we need to ship that application to the web. So what's next? How will the future of browser automation will look like? Honestly, I can't really tell for sure. I also will definitely not recommend you one framework over the other. I'm excited about all these developments that we have in the space. And even though I would like to see the WebDriver protocol as a standard to succeed, at the same time I'm rooting for tools like Cypress or Playwright that really push the boundaries of automation and really provide so much value for the developer persona. And honestly, without these frameworks, I wouldn't see that kind of development that we see that we have in the standard space right now. That said, if you want to improve the quality of the web, it doesn't go without standards, which is why I'm excited as well about the developments that happen in that space.

7. New Opportunities with WebDriver Protocol

Short description:

The new WebDriver protocol combines the best of all three approaches and provides opportunities for testing and automating web applications. Web standards are being shipped with WebDriver extensions, such as the web authentication API, HTML changes for time zone, and other APIs like sensor, browser reporting, and permission. The process of creating standards is slow but developing faster. Automation solutions based on proprietary protocols are limited, and test tooling should build on standards. Thank you for the opportunity to speak and for listening.

For one, we have this new WebDriver protocol in the making that enables you so much more opportunities. It really combines the best of all three approaches and creates tremendous chances for implementers to build tools that effectively can test and automate web applications. It is also very exciting to see that more and more web standards are shipped with WebDriver extensions. We have, for example, the web authentication API that allows you to create virtual authenticators with WebDriver. Very recently, we have seen changes in HTML that have caused the creation of a new WebDriver command to change the time zone. And we have much more APIs, like the sensor API, browser reporting APIs, and permission APIs that already have WebDriver extensions.

That said, creating any form of standards is usually a very slow process. You need to get people on board. You have to make sure that what you standardize makes sense for all users. And you have to write the spec. So, these changes won't happen overnight. It is really good to see though that all these developments in the tooling and standard space are taking place and are getting faster and faster.

We are at the end of my talk, and I wanted to end with the following quote by Maya and James, who are developers at Mozilla. They say, an automation solution based on a proprietary protocol will always be limited in the range of browsers it can support. The success of the web is built on multi-vendor standards. It is important that test tooling builds on standards as well, so that tests work across all the browsers and devices where the web works. With that, thank you so much for giving me this opportunity to speak, and thank you all for listening.

Hi, Christian. Thanks again for your talk. It was really good. Thank you. What do you think of the result of the polls? I mean, I was expecting that answer. Every time you get a technical question, it depends on the right way to go. And to answer it, in my opinion, it depends. I guess there are a lot of ways how you can automate the browsers. Right now, the most popular ones are usually down to three, which I explained in my talk. Yes. Yeah. Like I said, it depends. This is probably a much-used answer for a lot of questions.


Questions on Low-code Automation and Browser APIs

Short description:

Speaking of questions, I saw we had a few questions in the Q&A channel already. Triderman asked about low-code automation platforms like Tosca. I've been exploring low-code solutions recently and think it's a great way to introduce more people to testing. Sudharsan inquired about running browser tests in headless mode in Cloud run times. The stock broker asked about the future of native tools and browser APIs. Guy questioned whether tools can intelligently understand SPAs. The automation of browsers is asynchronous, and waiting for elements to exist is crucial. Tools that simplify this process will succeed as automation frameworks.

It just works. Speaking of questions, I saw we had a few questions in the Q&A channel already. Triderman, I'm so sorry if I'm pronouncing this wrong, asked, what are your thoughts on the low-code automation platforms like Tosca? Yeah. I've been exploring low-code solutions a lot recently as it becomes more and more interesting. And I think it's a great way to introduce more people to testing. And these tools are also becoming more and more sophisticated that really provide valuable solutions for people that just don't code. And that's totally fine. And we want to have those people also being able to test software. Yeah. In my experience, also having people with the knowledge about the product being able to help write the tests is invaluable. Okay. Yeah. Sudharsan, again, so sorry if I'm butchering the names, wrote, I prefer to run browser tests in headless mode in Cloud run times like Lambda-slash-Google functions. I would like to know which framework has native support. Do you know anything about this, Christian? I've been experimenting with that in the Web Developer Project to be able to just run a bunch of Lambda functions in the wild and scale up to like thousands without touching your own CPU. That's a really interesting thing, and I think that's something where we could explore to go to move forward, at least for people that might be a reliable solution that want to really test on one browser that runs in the docker container efficiently. So let's see what the future brings. Yeah, I've never done anything like this, but let's hope the future brings a lot more to support this. The stock broker asked how much time until native tools will Eclipse be emulated and browser APIs will rely on tools and in what ways? I'm not sure if I get this question right, how much time until native? So I think the browser, the browser APIs will continue to develop themselves as they are usually a good way to help people debug the browser as well. So we will definitely see a lot of development there across browsers and across browser engines. Yeah, I guess maybe the question was about usage or something, but maybe the stockbroker can clarify if this didn't answer his question. Guy said my experience with these frameworks is a lot of wait for x to exist. Is there a future where these tools can intelligently understand SPAs? Yeah, I've made that experience quite a lot myself. Yeah, I think that's a good question. The thing about automating a browser is that browsers in itself as this piece of software are really asynchronous. The idea of browser is a really asynchronous idea. And those methods where you have to wait for an element to exist is something that you need to do to make this browser automation work. There are tools that try to help you with that. And I think tools that are really good in this will succeed as an automation framework, because it's really difficult for people that don't develop browsers to have an understanding about that. And we should make this as simple as possible.

Development, Standardization, and Future

Short description:

There will be continued development across all frameworks. SmartTV and obscure browser vendors are not part of the current standardization efforts. The focus is on web protocols, with past attempts at mobile standardization. Getting vendors to agree on standards is challenging. The future may bring standardization in the IoT space. The web authentication extension may not support basic HTTP auth, but it's on the roadmap. Further questions can be addressed in the discussion room. Thank you for joining us.

And there will definitely be development like this. We see it across all the frameworks right now. I think this kind of development will be continued as well. Yeah.

Okay. Philly Fala said, are SmartTV vendors and other more obscure browser vendors also part of this standardization? No. Currently, the web protocols focused on the web. There have been some efforts in the past to create some sort of mobile standard around automation. The problem is that we need to get these vendors on board to sit on a table and agree on those kinds of standards. And as long as that doesn't happen, it will be difficult to create a standard like this for the web. We have all agreed to, all the browser vendors have agreed to work on those kinds of standards. I would love to see that in the IoT space as well. And we will see what the future brings.

Okay. I think we have time for one last question. Joey Halfrich asked, will the web authentication extension allow basic HTTP auth? We use it to keep the bots off of sites in development, but have to turn it off to run tests. I unfortunately haven't looked too much into that web developer extension. I'm not sure. I don't think it will allow the basic auth to happen, but it is something that is in the web developer Bidi roadmap for sure. Okay. I think all of the rest of the questions can be answered in the discussion room on, where Christian will join you shortly. Thank you so much again for joining us. It was a pleasure.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

TestJS Summit 2021TestJS Summit 2021
33 min
Network Requests with Cypress
Top Content
Whether you're testing your UI or API, Cypress gives you all the tools needed to work with and manage network requests. This intermediate-level task demonstrates how to use the cy.request and cy.intercept commands to execute, spy on, and stub network requests while testing your application in the browser. Learn how the commands work as well as use cases for each, including best practices for testing and mocking your network requests.
TestJS Summit 2021TestJS Summit 2021
38 min
Testing Pyramid Makes Little Sense, What We Can Use Instead
Top Content
Featured Video
The testing pyramid - the canonical shape of tests that defined what types of tests we need to write to make sure the app works - is ... obsolete. In this presentation, Roman Sandler and Gleb Bahmutov argue what the testing shape works better for today's web applications.
TestJS Summit 2022TestJS Summit 2022
27 min
Full-Circle Testing With Cypress
Top Content
Cypress has taken the world by storm by brining an easy to use tool for end to end testing. It’s capabilities have proven to be be useful for creating stable tests for frontend applications. But end to end testing is just a small part of testing efforts. What about your API? What about your components? Well, in my talk I would like to show you how we can start with end-to-end tests, go deeper with component testing and then move up to testing our API, circ
TestJS Summit 2022TestJS Summit 2022
20 min
Testing Web Applications with Playwright
Top Content
Testing is hard, testing takes time to learn and to write, and time is money. As developers we want to test. We know we should but we don't have time. So how can we get more developers to do testing? We can create better tools.Let me introduce you to Playwright - Reliable end-to-end cross browser testing for modern web apps, by Microsoft and fully open source. Playwright's codegen generates tests for you in JavaScript, TypeScript, Dot Net, Java or Python. Now you really have no excuses. It's time to play your tests wright.
TestJS Summit 2021TestJS Summit 2021
31 min
Test Effective Development
Top Content
Developers want to sleep tight knowing they didn't break production. Companies want to be efficient in order to meet their customer needs faster and to gain competitive advantage sooner. We ALL want to be cost effective... or shall I say... TEST EFFECTIVE!But how do we do that?Are the "unit" and "integration" terminology serves us right?Or is it time for a change? When should we use either strategy to maximize our "test effectiveness"?In this talk I'll show you a brand new way to think about cost effective testing with new strategies and new testing terms!It’s time to go DEEPER!

Workshops on related topic

React Summit 2023React Summit 2023
151 min
Designing Effective Tests With React Testing Library
Featured Workshop
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents- The different kinds of React application tests, and where component tests fit in- A mental model for thinking about the inputs and outputs of the components you test- Options for selecting DOM elements to verify and interact with them- The value of mocks and why they shouldn’t be avoided- The challenges with asynchrony in RTL tests and how to handle them
Prerequisites- Familiarity with building applications with React- Basic experience writing automated tests with Jest or another unit testing framework- You do not need any experience with React Testing Library- Machine setup: Node LTS, Yarn
TestJS Summit 2022TestJS Summit 2022
146 min
How to Start With Cypress
Featured WorkshopFree
The web has evolved. Finally, testing has also. Cypress is a modern testing tool that answers the testing needs of modern web applications. It has been gaining a lot of traction in the last couple of years, gaining worldwide popularity. If you have been waiting to learn Cypress, wait no more! Filip Hric will guide you through the first steps on how to start using Cypress and set up a project on your own. The good news is, learning Cypress is incredibly easy. You'll write your first test in no time, and then you'll discover how to write a full end-to-end test for a modern web application. You'll learn the core concepts like retry-ability. Discover how to work and interact with your application and learn how to combine API and UI tests. Throughout this whole workshop, we will write code and do practical exercises. You will leave with a hands-on experience that you can translate to your own project.
React Summit 2022React Summit 2022
117 min
Detox 101: How to write stable end-to-end tests for your React Native application
Top Content
Compared to unit testing, end-to-end testing aims to interact with your application just like a real user. And as we all know it can be pretty challenging. Especially when we talk about Mobile applications.
Tests rely on many conditions and are considered to be slow and flaky. On the other hand - end-to-end tests can give the greatest confidence that your app is working. And if done right - can become an amazing tool for boosting developer velocity.
Detox is a gray-box end-to-end testing framework for mobile apps. Developed by Wix to solve the problem of slowness and flakiness and used by React Native itself as its E2E testing tool.
Join me on this workshop to learn how to make your mobile end-to-end tests with Detox rock.
Prerequisites- iOS/Android: MacOS Catalina or newer- Android only: Linux- Install before the workshop
TestJS Summit 2023TestJS Summit 2023
48 min
API Testing with Postman Workshop
In the ever-evolving landscape of software development, ensuring the reliability and functionality of APIs has become paramount. "API Testing with Postman" is a comprehensive workshop designed to equip participants with the knowledge and skills needed to excel in API testing using Postman, a powerful tool widely adopted by professionals in the field. This workshop delves into the fundamentals of API testing, progresses to advanced testing techniques, and explores automation, performance testing, and multi-protocol support, providing attendees with a holistic understanding of API testing with Postman.
1. Welcome to Postman- Explaining the Postman User Interface (UI)2. Workspace and Collections Collaboration- Understanding Workspaces and their role in collaboration- Exploring the concept of Collections for organizing and executing API requests3. Introduction to API Testing- Covering the basics of API testing and its significance4. Variable Management- Managing environment, global, and collection variables- Utilizing scripting snippets for dynamic data5. Building Testing Workflows- Creating effective testing workflows for comprehensive testing- Utilizing the Collection Runner for test execution- Introduction to Postbot for automated testing6. Advanced Testing- Contract Testing for ensuring API contracts- Using Mock Servers for effective testing- Maximizing productivity with Collection/Workspace templates- Integration Testing and Regression Testing strategies7. Automation with Postman- Leveraging the Postman CLI for automation- Scheduled Runs for regular testing- Integrating Postman into CI/CD pipelines8. Performance Testing- Demonstrating performance testing capabilities (showing the desktop client)- Synchronizing tests with VS Code for streamlined development9. Exploring Advanced Features - Working with Multiple Protocols: GraphQL, gRPC, and more
Join us for this workshop to unlock the full potential of Postman for API testing, streamline your testing processes, and enhance the quality and reliability of your software. Whether you're a beginner or an experienced tester, this workshop will equip you with the skills needed to excel in API testing with Postman.
TestJS Summit - January, 2021TestJS Summit - January, 2021
173 min
Testing Web Applications Using Cypress
This workshop will teach you the basics of writing useful end-to-end tests using Cypress Test Runner.
We will cover writing tests, covering every application feature, structuring tests, intercepting network requests, and setting up the backend data.
Anyone who knows JavaScript programming language and has NPM installed would be able to follow along.
TestJS Summit 2023TestJS Summit 2023
148 min
Best Practices for Writing and Debugging Cypress Tests
You probably know the story. You’ve created a couple of tests, and since you are using Cypress, you’ve done this pretty quickly. Seems like nothing is stopping you, but then – failed test. It wasn’t the app, wasn’t an error, the test was… flaky? Well yes. Test design is important no matter what tool you will use, Cypress included. The good news is that Cypress has a couple of tools behind its belt that can help you out. Join me on my workshop, where I’ll guide you away from the valley of anti-patterns into the fields of evergreen, stable tests. We’ll talk about common mistakes when writing your test as well as debug and unveil underlying problems. All with the goal of avoiding flakiness, and designing stable test.