Tiny Tests, Large Results

Rate this content

Yes, Big things do come in small packages. For example, isn’t a unit test’s speed, feedback, and reliability fantastic? Did you know we can also have fast, focused, and reliable feedback from our functional e2e tests? Atomic e2e tests are those that are targeted and focused. They’re tiny in size but large in their impact. This tutorial will teach you how to create atomic e2e tests with several code examples. First, we will use Cypress.io to authenticate by setting a cookie. Instead of using a UI. Second, we will use Cypress.io to set a JSON Web Token for authentication. Join me, and let’s write tiny tests for large results.

21 min
03 Nov, 2022


Sign in or register to post your comment.

Video Summary and Transcription

Automated atomic tests are a great way to improve UI tests by making them less brittle and faster. The tests focus on testing a single feature or component and have minimal UI interactions. The Talk explores examples of automated atomic tests and their implementation on web applications. It also discusses the analysis of atomic tests, login tests, and working with JSON Web Tokens for authentication and authorization. The Talk concludes by highlighting the use of UI and web requests in automated atomic testing.

Available in Español

1. Introduction to Automated Atomic Tests

Short description:

Automated atomic tests are a wonderful way to update your UI tests and make them less brittle and drastically faster. We're going to see a few examples of those automated atomic tests, and we're going to even implement those automated atomic tests on two web applications. An automated atomic test is a test that tests only a single feature or a component. They typically have very few UI interactions and typically touch a maximum of two screens, especially with front-end UI automation.

In today's tutorial, we're going to talk about automated atomic tests. Automated atomic tests are a wonderful way to update your UI tests and make them less brittle and drastically faster. We're going to see a few examples of those automated atomic tests, and we're going to even implement those automated atomic tests on two web applications.

One application will have an HTML web form and another will use a JSON web token for authentication. My name is Nikolay Advolatkin, Senior Solutions Architect at Sauce Labs and founder at ultimateQA.com. So, what are you waiting for? Let's go and check out how to implement automated atomic tests.

An automated atomic test is a test that tests only a single feature or a component. They typically have very few UI interactions and typically touch a maximum of two screens, especially with front-end UI automation. The reason why I say they typically touch a maximum of two screens is because these automated atomic tests will typically have a setup state where they need to set up the state of an application and then a post setup state where they need to perform a validation. Automated atomic tests have several good environments in that they are much faster and much more stable than your typical automated tests due to the fact that they decrease the amount of latency that we have to deal with and they decrease the amount of UI interactions that we have to perform.

2. Analyzing Automated Atomic Tests

Short description:

This test might look atomic at first, but it's not. It requires logging in and validating the login, which can be done separately. Atomic tests can be simple, like validating link attributes. We'll now look at an HTML web form application and a positive test case called 'Redirects to Dashboard on Success'. The test visits the login page, interacts with the form, and performs assertions to validate the login.

Now I got a question for you. This test over here, if you look at it and analyze it, do you believe this test is atomic? At first glance, this test might actually look atomic, right? In that it's signing in, validating that we were able to sign in and then it adds an item to a cart and asserts that an item was added to a cart. It's close to being atomic, however it's not actually atomic. The reason why it's not is because it has to login and validate that we have successfully logged in. There is no reason why we can't test the UI login in a separate test and ensure that works and then afterwards, what this test really cares about, what it actually wants to test is this portion right here. And so we can save time and stability by performing these interactions without using the user interface.

Atomic tests can come in many forms. Some of them can be extremely simple. For example, when we used to write automated atomic tests that click links, we actually have a wasted action and a wasted validation in that we need to click a link and assert that the link goes to the right location. What we're actually interested in is the Ahrefs attribute of the link, which we could easily validate in this manner, making our test faster and less brittle.

Let's go ahead and take a look at our first web application, which is going to have an HTML web form, and we're going to be able to log in to this HTML web form. But instead of using the UI, which is not efficient and not atomic, we're going to be able to make a web request that's going to end dropping a cookie in our browser, and then we're going to be able to bypass the log in without interacting with the UI. If we come to this application, there are many tests that are here, but I'm going to focus on one at a time, just to give you a better understanding. The very easiest test to understand, which is the positive test case, is going to be this test case here called Redirects to Dashboard on Success.

This is the very positive test case where we visit the login page, and by the way, you can see our application on the right hand side, if you're not familiar with Cypress. It displays the test execution on the left, with the actual application on the right, and if you did want to explore the application in another browser, you definitely can. Here it is, and it performs all the same exact operations, but with Cypress, for example, it's very easy in that you can see what the test is doing and then see the actual application on the right hand side and even interact with it if you wanted. You can see that the very first step that this test does is it visits the forward slash login of our application that is on local host 7077, and after that, it performs the standard operations that you would expect in order to actually be able to log into the application, right? So it's going to get the username and type in Jane Lane. It's going to get the password field and type in password and then it's going to submit the form. Once the form is submitted, you can see we do a couple of assertions. First, that the dashboard page is there, right? So our URL includes dashboard forward slash dashboard. So that means that we were able to successfully log in. Second, we get the header and assert that it contains Jane Lane. So almost checking not only that we're logged in, but that contains the right user and we get the cookie that corresponds to our login and make sure that there is a cookie that exists. So take a look at the application tab here and in our cookies. Currently, there is nothing available here. I've cleared it. But if we rerun the test, and of course, as I mentioned, it does interact with the application actually logged in. You can see that now there's a Cypher session cookie that's dropped, hence why we are able to log in through the application. The challenge with this test is that, again, it's not atomic, right? In that it does interact with this login form, and then it logs in, and then it validates that we're logged in.

3. Analyzing Login Tests

Short description:

If you wanted to do any kind of actions beyond the login, it would be repetitive and unnecessary. The tests help us understand the functionality of the application without manually stepping through it. The first test, 'unauthorized', redirects to the dashboard if not logged in. It checks for the 'unauthorized' message and the URL. If we try to log in without a cookie, we expect a 302 response with 'unauthorized' in the body. If we log in with an invalid username and valid password, we expect an error message. On success, we can visit the dashboard. This test is atomic and validates the login. We make the login atomic by submitting a web request to the login URL with the valid username and password, which sets the session cookie.

So that's great. But if you wanted to do any kind of actions beyond this, for the login, we'd be doing the same thing over and over every single time. And so it's repetitive and unnecessary, and maybe we can make that action more atomic. But let me actually show you the rest of the tests, so that you better get an understanding of what the application can do.

So I uncommented the only part of the test. And so now we have all of the tests that are available here. And these tests, of course, as good end-to-end tests or any kind of test should do, they kind of help us to understand the functionality of the application without actually manually stepping through it.

So the very first test here is called unauthorized. And so the expectation here is that you're redirected on the visit to forward slash dashboard. So if you're not logged in, you should be redirected. And so this is what this test does. It tries to visit dashboard, but then it gets redirected to you're not logged in and cannot access this page. Hence there's an assertion that gets the H3 and checks for that message. And it checks the URL includes unauthorized in it, right? So if you're not logged in, can't access the dashboard page. If we try to log in by doing a get on the dashboard page, we will also be unable to do that. And so we're expecting a 302 to come back to us as a result when we try to log in without a cookie. And we'll also expect the response to come back with unauthorized in the body.

If you try to interact with the actual form and you try to log into it with, for example, invalid user and the valid password, we expect there an error to be visible and the error to say that username and or password is incorrect. And on Success, you've already seen this. If we visit the dashboard, type in the valid user and the valid password, we get a success in that you're able to visit the dashboard by itself. This test is atomic and great and very useful to test a single time. Once you're able to log in with the web form through the UI, you know that it's going to work and it no longer needs to repeat it in the rest of the tests.

And so how do we make it atomic? In this case, for this web application with an HTML web form, the way that it works is you submit a request. How would an atomic log in actually look under the hood? Well, as you can see from the documentation of this test here, we can simply perform a web request that will allow us to log in. The web request is simply going to be a post to the log in URL, passing in the valid username and password. And then that will automatically set the session cookie and so now after this point, we can perform any operations that we want that are behind the private application interface. So, let me show you what exactly that looks like. And so, here you can see doing the post to the log in, right? And then showing you that the cookie does exist. And in fact, we can even simply do, just take a look at this test so that it's focused. Go back here to the application, get rid of this and then if we rerun it, we're going to be able to see that there is a cookie.

4. Working with JSON Web Tokens

Short description:

We're going to work with a Vue application that uses JSON web tokens for authentication. JSON web tokens are commonly used in many applications to authenticate and provide access based on the token. We'll submit a request with username and password, and the application will verify them against the database. We'll receive a token that can be used for authentication. This is extremely useful for automated tests. We'll be working out of the 'logging-in JWT' folder in the larger Cypress examples repo. Let's take a look at the tests to understand how they work with the UI and in an atomic manner.

And nothing happened in our application, right, but if for example now, we come and try to access the dashboard because we are logged in and we have a cookie in our browser, we're able to access it without actually logging in. This time instead of our application being an HTML web form, it's actually going to use JSON web tokens. JSON web tokens are a very common industry standard method of basically authenticating into applications and allowing access to certain parts of the application based on this web token. If you want to learn more about it, you can go to this website here JWT.IO, but basically the way it works is you submit a request with your username and access key or username and password and then based on that, the application will verify whether those exist in the database. You get back a token that looks like this and now this token is what you can use for authentication, very common in many applications in today's day and age. So extremely useful to be able to do this in our automated test. Here is the repo that we're going to be working out of. As before it's very similar. This folder that I'm in is part of a much larger folder. I can show you exactly here. So you can see the folder I'm in is logging-in JWT. If I navigate up, you can see I'm in the larger Cypress examples repo. And then as I said, we're working out of logging in JSON web token folder. The application we're going to be working on is a Vue application, and you see the instructions on how to start the app here, which I did over here. Actually, I simply ran NPM run start to start the application. And so that's running in here. And then in another terminal window, I ran NPM run Cypress open, and that opened up my Cypress browser. And so now, let's go take a look at the tests to see exactly what they're doing and how to do it through the UI, and then how to do it without a UI through an atomic manner.

5. Working with Authorization and Tokens

Short description:

The app stores login credentials in local storage and requires an authorization header with a bearer token for accessing resources. The network tab shows the authorization and bearer token with the JSON Web Token.

The way this app is going to work is it's going to send the login credentials, and then those login credentials are going to be stored in local storage in an item named user. You can see from this screenshot here, there's our token. And then any requests that we make afterwards that are behind this authorized information, that's going to require the authorization header with a bear token in order to be able to access the resources. And so if we, for example, take a look at the network tab here under the users, you'll see that there is an authorization and a bear token with the JSON Web Token, and that's how users can get access to different parts of the application.

6. Automated Atomic Tests with UI and Web Requests

Short description:

Once Cypress is up and running, we can perform tests using UI and web requests. The UI tests involve logging in, validating the login, and performing various assertions. We can also test unauthorized access and invalid login attempts. By using web requests, we can make the tests atomic and bypass the need for UI interactions. We authenticate the user, set the necessary token, and access the user's data. We can then check for different conditions. This demonstrates the use of JSON Web Token login mechanism. There are various login mechanisms available, and the relevant one can be chosen based on the application. Bonus resources on automated atomic testing are available in the ReadMe. Thank you for joining this tutorial and feel free to connect with me on ultimateQA.com or my YouTube channel.

Once Cypress is up and running, we'll be able to see these two specs here, right? One is using UI and then the other one is not. And so if we pull up this using UI spec, you'll see all of the tests that are performed, right? Using UI, it'll be something very standard. You've seen before, which is basically getting the username, typing in that username, typing in the password into the password field, and then clicking the login button and you can see what happens before and after. Afterwards, we are basically checking here. You can see there's a web request happening of a Pulse 200 and then we're doing a number of operations such as we're making sure that we are at the correct URL, expecting this header to be visible, and then we're even expecting that there is an item with a username first name to a last name and then it has this token that is a JSON web token and we're expecting it to be a string. And then there's a bunch more assertions on different parts of the web request. There's even more assertions like this checking that there's a logout link and then clicking the logout link and logging out and asserting that we're on the login page. So if you try to access the localhost 4000 slash users, this URL, you will get unauthorized. Right? So you get this error message. If we come here and try to access this application invalid token because we can't get there. And so we're supposed to get back a 401. Right? So if we come here and inspect and come here to the network, and then we do Yelp XHR and the users, we will get here a 401. You can see the users document returned was 401 because you're unauthorized. And so this basically obviously shows a positive test case and a negative test case both through the UI. Oh, and here is actually one with the UI that will try to log in with an invalid username. Right? So typing in username, typing in a wrong password, and then clicking to log in. And then of course, it's going to get that username or password is incorrect. And hence it's doing that assertion to make sure that it exists. So that's our UI test. Wonderful. Again, fast on local host will be much more cumbersome when we actually deploy this to an environment that have to travel over the network to interact with this and more flaky as well as a result. So, hence we can bypass all this using web requests as we did with the previous web application. So if we wanted to make the same test atomic, we have these specs here and I'm going to run them. Unfortunately, they're not as useful to see through the Cypress UI because they do make a bunch of web requests. And so the information behind them is not super useful, but you can see how we can make an authenticated request, make sure that we're logged in, and show a loaded user. So instead we can actually look at the code to understand how to make this possible with JWT. So if we look at this spec.cy.js, we can see that the very first step that we do is actually hit an endpoint to authenticate a user with a username and password, and then store the returning body inside of the user. Then, in order to be able to add the item to our local storage, we need to set that item called user and we stringify that user, hence enabling that Bare Token. And so, once that's set, we can try to hit this endpoint with this Bare Token and we are able to access this user, without which, right now, we would not be able to access it. But with it, we are. And then, of course, as a result, we can check for different kinds of conditions, to be or not to be visible. And that's it. We've created an automated atomic test using a JSON Web Token login mechanism. Now, there are actually lots of different types of login mechanisms and certain ones may be relevant to your application, others may not. There are many more examples here, as you'll see in this repo. You'll see all of these login examples right here, from Basic Auth, all the way to Single Sign-on, so depending on your application you'll use the relevant login mechanism. Also, as a bonus, I've linked a bunch of bonus resources to how to do automated atomic testing in the ReadMe, it's at the bottom of the ReadMe, so definitely be sure to check that out. Thank you so much for joining me in today's tutorial about automated atomic tests, and if you want to learn more about me or catch up with me, you can find me at ultimateQA.com or my YouTube channel or any of these other social medias down below. Thanks again so much for your time and I'll see you next time.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

TestJS Summit 2021TestJS Summit 2021
33 min
Network Requests with Cypress
Top Content
Whether you're testing your UI or API, Cypress gives you all the tools needed to work with and manage network requests. This intermediate-level task demonstrates how to use the cy.request and cy.intercept commands to execute, spy on, and stub network requests while testing your application in the browser. Learn how the commands work as well as use cases for each, including best practices for testing and mocking your network requests.
TestJS Summit 2022TestJS Summit 2022
27 min
Full-Circle Testing With Cypress
Top Content
Cypress has taken the world by storm by brining an easy to use tool for end to end testing. It’s capabilities have proven to be be useful for creating stable tests for frontend applications. But end to end testing is just a small part of testing efforts. What about your API? What about your components? Well, in my talk I would like to show you how we can start with end-to-end tests, go deeper with component testing and then move up to testing our API, circ
TestJS Summit 2022TestJS Summit 2022
20 min
Testing Web Applications with Playwright
Top Content
Testing is hard, testing takes time to learn and to write, and time is money. As developers we want to test. We know we should but we don't have time. So how can we get more developers to do testing? We can create better tools.Let me introduce you to Playwright - Reliable end-to-end cross browser testing for modern web apps, by Microsoft and fully open source. Playwright's codegen generates tests for you in JavaScript, TypeScript, Dot Net, Java or Python. Now you really have no excuses. It's time to play your tests wright.
TestJS Summit 2023TestJS Summit 2023
21 min
Everyone Can Easily Write Tests
Let’s take a look at how Playwright can help you get your end to end tests written with tools like Codegen that generate tests on user interaction. Let’s explore UI mode for a better developer experience and then go over some tips to make sure you don’t have flakey tests. Then let’s talk about how to get your tests up and running on CI, debugging on CI and scaling using shards.
TestJS Summit 2023TestJS Summit 2023
29 min
Fighting Test Flakiness with Time Machines
What would you do differently if you could travel back in time? Modern testing frameworks have transformed this whimsical question into a practical one, by creating their own “time machines”.Cypress’ timeline, Playwright’s trace-viewer and Replay.io’s recordings have offered a retrospective look into the life of a test, ensuring that developers and testers are no longer limited to basic error messages on test failures.However, these different time machines will bring different insights. So how do you decide? The decision on which one to use can make a significant difference in time spent on debugging a flaky test.In this presentation I will be focusing on comparing different time machine solutions and showing various flaky test examples to demonstrate how to navigate through debugging process and believe it or not - make it fun.Key Takeaways:- learn about how different time machine solutions work- discover how to effectively use time machines to debug a flaky test- find out about sources of flakiness within the test and within the application under test
TestJS Summit 2022TestJS Summit 2022
17 min
Testing Mail Service With Playwright
Top Content
We send emails to our users - account verification and newsletters. We allow the user to contact us by sending an email via inbuild form. Do we? Does the user receive an account verification email or exactly what notification they signed up for? We can cover this functionality as part of E2E tests: get an email and open it to check what is in it. We will need Playwright and a fake SMTP server to capture emails sent by the app.

Workshops on related topic

TestJS Summit 2022TestJS Summit 2022
146 min
How to Start With Cypress
Featured WorkshopFree
The web has evolved. Finally, testing has also. Cypress is a modern testing tool that answers the testing needs of modern web applications. It has been gaining a lot of traction in the last couple of years, gaining worldwide popularity. If you have been waiting to learn Cypress, wait no more! Filip Hric will guide you through the first steps on how to start using Cypress and set up a project on your own. The good news is, learning Cypress is incredibly easy. You'll write your first test in no time, and then you'll discover how to write a full end-to-end test for a modern web application. You'll learn the core concepts like retry-ability. Discover how to work and interact with your application and learn how to combine API and UI tests. Throughout this whole workshop, we will write code and do practical exercises. You will leave with a hands-on experience that you can translate to your own project.
React Summit 2022React Summit 2022
117 min
Detox 101: How to write stable end-to-end tests for your React Native application
Top Content
Compared to unit testing, end-to-end testing aims to interact with your application just like a real user. And as we all know it can be pretty challenging. Especially when we talk about Mobile applications.
Tests rely on many conditions and are considered to be slow and flaky. On the other hand - end-to-end tests can give the greatest confidence that your app is working. And if done right - can become an amazing tool for boosting developer velocity.
Detox is a gray-box end-to-end testing framework for mobile apps. Developed by Wix to solve the problem of slowness and flakiness and used by React Native itself as its E2E testing tool.
Join me on this workshop to learn how to make your mobile end-to-end tests with Detox rock.
Prerequisites- iOS/Android: MacOS Catalina or newer- Android only: Linux- Install before the workshop
TestJS Summit - January, 2021TestJS Summit - January, 2021
173 min
Testing Web Applications Using Cypress
This workshop will teach you the basics of writing useful end-to-end tests using Cypress Test Runner.
We will cover writing tests, covering every application feature, structuring tests, intercepting network requests, and setting up the backend data.
Anyone who knows JavaScript programming language and has NPM installed would be able to follow along.
TestJS Summit 2023TestJS Summit 2023
148 min
Best Practices for Writing and Debugging Cypress Tests
You probably know the story. You’ve created a couple of tests, and since you are using Cypress, you’ve done this pretty quickly. Seems like nothing is stopping you, but then – failed test. It wasn’t the app, wasn’t an error, the test was… flaky? Well yes. Test design is important no matter what tool you will use, Cypress included. The good news is that Cypress has a couple of tools behind its belt that can help you out. Join me on my workshop, where I’ll guide you away from the valley of anti-patterns into the fields of evergreen, stable tests. We’ll talk about common mistakes when writing your test as well as debug and unveil underlying problems. All with the goal of avoiding flakiness, and designing stable test.
TestJS Summit 2023TestJS Summit 2023
89 min
Building out a meaningful test suite that's not all E2E
We're all taught to follow the Testing Pyramid but the reality is that we build out the Testing Christmas Tree. In this workshop, David will talk you through how to break down projects and put the tests where they need to be. By the end of the workshop you will be able to update your projects so that anyone and everyone can start contributing and truly living up to "Quality is everyone job".
He will walk you through:- Component Testing- API Testing- Visual Regression Testing- A11Y testing
He will also talk you through how to get these all setup in your CI/CD pipeline so that you can get shorter and faster feedback loops.
TestJS Summit 2021TestJS Summit 2021
114 min
Flaky Test Management with Cypress
This workshop is for Cypress users who want to step up their game against flake in their test suites. Leveraging the Cypress Real World App, we’ll cover the most common causes of flake, code through some examples of how to make tests more flake resistant, and review best practices for detecting and mitigating flake to increase confidence and reliability.

Table of contents:
- Cypress Real World App Overview
- What is Flake?
- Causes of Flake
- Managing Network-related Flake (Activity)
- Managing Dom-relate Flake (Activity)
- Flake Detection and Mitigation Best Practices
- Q&A