Dissecting Complexity in Tests

Rate this content

Learn about the most common reasons for complexity in tests, how it manifests, and how to deal with that complexity to produce elegant tests even for the most complex systems.

15 min
03 Nov, 2022


Sign in or register to post your comment.

AI Generated Video Summary

Today's Talk discusses complexity in tests and how to effectively deal with it. The speaker emphasizes the importance of testing critical user-facing paths and modeling tests from the user's perspective. They also highlight the significance of creating a testing setup that allows any test to run smoothly and the implicit testability of a well-designed system. The Talk explores the impact of choosing the right testing environment, the role of testing setup in mitigating complexity, and the importance of test structure and expectations. The speaker provides practical tips for tackling complexity in tests, such as keeping tests flat, using helper utilities, and splitting tests into separate files.

1. Introduction to Complexity in Tests

Short description:

Today, I would like to speak about complexity in tests. Complexity is destined to happen, but it's how we choose to deal with it that matters. Complexity in tests can come from the system being tested or from the tests themselves. When dealing with complexity from the system, start by testing the most critical user-facing paths. When it comes to testing, model it from the user's perspective and invest enough into testing setup.

Hello, everyone. My name is Artem, and I'm a software engineer at Kotlinbox. Today I would like to speak about complexity in tests, but before I begin, allow me to ask you a simple question. Have you ever felt like writing a test for a feature would require more time and effort than the feature itself? Well, like me, you have. Then, chances are you were dealing with one or maybe multiple ways how complexity may manifest in your code base.

But you shouldn't feel bad about it, because no matter how great engineers we are and what incredible code we write, complexity is destined to happen. It's fine. Complexity in itself is not the issue. It's how we choose to deal or not to deal with how it manifests what matters. And while complexity can be a broad topic, for the sake of today's talk, I would like to reference to it as a quality or state of being hard to write, understand, and maintain a test. And when it comes to complexity in tests, it can be divided into two main groups. It's the complexity that comes from the system that we're testing, and this can be really any code. A React component, a backend route handler, or a JavaScript library. And complexity that comes from the tests that we're writing.

So let's start from the system. And one of the most common ways how people stumble upon complexity, coming from the code that they test, is that they don't know what to test. I'm pretty sure you've been in this situation. You open an existing file and it seems to be doing everything possible in the universe, and you have zero idea how to even approach testing that. Well, there's actually a great rule you can follow in these situations. It is, whenever you're in doubt, start by testing the most critical user-facing paths. So if you're building an e-commerce product, well, starting a test strategy from a logging flow or a checkout flow makes the most sense. And if you're developing internal tooling or libraries, then start from those happy paths that users expect, and that should set you on the right track.

And then, when you know what to test, the next biggest problem, the next challenge, is how to test that. And I think very often, when we feel struggle with how to approach testing, it's because we may miss some sort of testing philosophy. And one of the most useful approaches that I've adopted over the years is testing like the user. What it means is that whenever you write a test, try to model it from the user's perspective. So your test actions that you perform would emulate the actions that that user would do with your software. And your assertions that you write actually reflect user expectations as the result to their actions. And then another thing that helps tremendously is when you invest enough into testing setup. And I feel this is very often overlooked and it's a shame, because the testing setup is perhaps one of the most important phases that deals with the complexity.

2. Complexity in Tests: Purpose and Testability

Short description:

The point of this phase is to create a universe where any test can run without problems. Each test should have a purpose, which is to describe the intention behind the system. Testing the testability of a system is an implicit test in itself. Poorly designed systems are hard to test, while well-designed systems make testing easier.

Because the point of this phase is to create this universe, this box, where any test can run, or any test that you want to write can actually execute without problems. And I'm going to talk about testing setup a little bit later into the talk.

Okay, so when you know what to test and how to test, you may be stumbling into another problem that is writing too many tests. And it may sound like a good thing at first, but it's not really because each test should have a purpose. And we often seem to forget the purpose behind testing in general.

And we write tests not to gain code coverage or to have the CI passing, although we want that. We actually write tests for a single reason. And that is we write tests to describe intention behind the system. Think about it. Whenever you write a piece of logic in your code, you have some sort of intention. You want that code to do something. But unless you have an automated test to validate that intention, you have no proof that your code works as expected.

So the next time you approach a test, ask yourself a question. Is what I'm testing actually related to the intention behind this code? Because if it's not, chances are you can drop this test and still lose no value in your testing setup.

And then the other thing is that, well, real world is quite more complex than that. And sometimes they're objectively complex systems, right? Or are there? Because one thing I love about testing is that a testability of the system is an implicit test in itself.

Now, what this means is that when you have poorly designed, poorly architectured systems, as a consequence, they're going to be really hard to test. And the opposite stands true also. Let me give you a few examples of how this manifests.

So in this get user function, we fetch user from the database. But we also fetch all the post for the user. And this feels that it doesn't belong here. Because now to properly test this function, we need to also mock everything related to posts. And this is a stretch. What maybe the proper approach here would be to split this one function into two and test them all in isolation, which would be much easier.

Another example is related to dependencies that our code introduces. Like this shopping cart controller. You can see that in the constructor, we're creating a new database connection. Well maybe that's not a good idea because to test this controller now, we need to implicitly mock this database constructor somehow. Why not just pass it as an argument to the constructor, do dependency injection, and thus allow us to test, for example, against the test database during testing, which would make this whole experience much easier.

3. Tackling Complexity in System Testing

Short description:

Stick to best practices to write better code and improve tests. Establish a clear testing strategy and focus on critical user-facing paths. Test like the user and invest in the testing setup. Use testability as an implicit check for code intention.

But my point here is not to give you some practical pieces of advice on how to write better code. I'm pretty sure you know that already. I'm just trying to encourage you to stick to those best practices. Because the better code you write, the better tests are going to be for that code. So best practices matter.

So to summarize. How do we tackle complexity that comes from the system? First of all, we establish a clear testing strategy. And when we're in doubt, we're testing the most critical user-facing paths. Then we adopt some philosophy, for example, like testing like the user, which really helps us model our tests easier and we know how to approach any logic that we test. We need to invest into the testing setup because it's one of the most important parts of the setup that allows us to write any test we need. And of course, we can use the testability as sort of an implicit check to help us see that the code that we're writing still stands true to the intention that we have for that code.

4. Introduction to Testing Complexity

Short description:

We often introduce complexity in tests ourselves by choosing the wrong testing environment. It's important to select the proper testing environment that aligns with the code's intended runtime. For Next.js pages, automate tests in a browser, while for simple JavaScript functions, a Node.js-based testing framework would suffice.

Okay, now let's speak about complexity in tests. And one thing that often comes to mind is that we introduce that complexity ourselves. For example, when choosing the wrong testing environment. Imagine if you're testing a Next.js page, but you decided to do it in GSDOM. Well, you're gonna have really bad time because that environment is not designed to test full pages. The same is true if you decide to test a single JavaScript function, and you spawn an entire Chromium instance to do that. Sure, that would work, but is that really the right approach? And to solve this is really straightforward. Choose the proper testing environment, and it is often the environment in which your code is destined to be run. So if it's Next.js page, just launch it in a browser and automate tests there. If it's a simple JavaScript function, perhaps a Node.js-based testing framework would be enough to test it efficiently.

5. Testing Setup and Assertions

Short description:

The testing setup is crucial in mitigating complexity. It creates an environment where any test can run and handles side effects and common code abstractions. Tackle complexity in the setup phase and use helper functions to reduce visual clutter. Avoid complexity in assertions. Keep the testing setup simple and respond to complexity with granularity. Overdoing assertions can lead to complexity and repetitiveness.

Okay, now, the other thing, and it's perhaps one of the most crucial things in the whole testing section, is the testing setup. And how very often we lack it. I mentioned it briefly before, so let's go into more detail here. And the idea behind the testing setup is to create this environment where any test that you need can run. And this is why this phase should do the most heavy lifting in terms of complexity mitigation.

So this is where you do HTTP request mocking, where you create test databases or mock connection to databases and tackle any sort of side effects that your code commonly introduces. This is why utilizing setup and action phases as well is crucial. And let me show you how using an example. So when it comes to mitigating complexity, you really want to do the most of it in the setup phase, like I've mentioned. One of the reasons why is because you do it once and you have this environment and you can run any test in it, which is great. But then even after that, you're still going to have some occasional complexity coming from the test actions that you perform, because there's very often some logic, some abstractions that we do in testing, and you can just move them into helper functions and utilities and reduce the visual clutter, but also complexity of tests overall. And you absolutely never, ever want to tackle complexity on the assertions level. And I'm going to show you an example why in just a minute.

But most importantly, keep it simple. I once had the pleasure of reviewing a pull request that had a goal to improve the testing setup. And while it was great, I'm ashamed to admit that it took me around 25-30 minutes just to understand what a test setup was doing for a single test. So half an hour, but I wasn't anywhere near understanding what the test does, what the code does behind the test. No, just the setup. And it's really important to keep this in mind when tackling complexity. You shouldn't really respond to complexity with more complexity because math still stands and one plus one can equal two complexities. Instead, you want to respond to complexity with granularity. So single purpose small functions that in total contribute to creating the testing setup that you need.

Okay, now let's talk about assertions, I think in the light of reducing complexity and repetitiveness, we tend to overdo it sometimes. And here's an example for you. So this is an assertion from a test block. And whenever I read any test, I'm actually starting from this expect lines because they're the most useful to me. So in this test, we expect the file content to equal a string. This is pretty straightforward. But it's not what the test does. Because if we look above this expect line, we see that there's a for loop.

6. Complexity in Test Structure and Expectations

Short description:

Testing every file from a follower to have the same content, testing all the followers and their files to equal a specific string, abstracting complexity, improving the expect line, focusing on equality, and the importance of test structure.

So we're actually testing every file from a follower to have the same content. And even that is not enough because there's another loop above. And we actually testing all the followers and all their files to equal a specific string. Just notice how many things we need to compute in our head just to understand what the single expect line does.

So it feels like we abstracting complexity, but we actually just adding more complexity to our minds to tackle. So I believe that a test block is the worst place to get smart. And let me show you how we can redo this expect line to be much better. So this is the same assertion from before, but now it reads in a single line. We expect that all file contents would equal a string. That's all. There's no additional context attached. And if we need to find out where the file contents are coming from, we can just jump to the line that gets them, and we can see, hey, it uses a utility function. So we abstract that logic away because in actuality, this test doesn't concern itself with how to extract those contents. It only concerns itself about the equality.

And then another point relates to test structure. And let me tell you a story. I was once working on a really big project, and it had a lot of tests. And one of the tests used was 4,000 lines of code long. And as it often happens, something went wrong. There was an issue, and CI started to fail. So I jumped into this and tried to figure out what's happening. And I saw that this test was failing. This assertion was failing. And I spent a couple of minutes, and half an hour, then an hour, and it just didn't make sense to me. Because, well, it was like saying I expect 1 to equal 1, and it was false. It didn't make sense. But then I finally figured out that a couple of thousand lines above that failing assertion, before OBLOCK, it was completely mutating the result of the whole system, and I wasn't very happy about it. But it taught me an important rule. Is that we should try writing tests that would still make sense at 3am. Because, imagine, it's middle of the night, and pager duty wakes you up, and production is failing.

7. Dealing with Complexity in Tests

Short description:

To prevent the frustration of debugging complex tests, keep them flat and eliminate unnecessary describe blocks. Use helper utilities to abstract common logic, such as filling in a signing form. Split tests into separate files for readability and maintainability. Properly utilize test phases and reduce repetition. Keep test structure flat and use simple explicit assertions. Split complex features on the file system level for better discoverability and maintenance.

So you open your laptop, and the first thing you do, you go to the test, which is hopefully there, and you try to figure out what's failing, and what is the intention, how it's supposed to work. But if you have a lot of smart assertions to compute in your head, if you had a complicated testing setup, if you have this mutable-system result, you're gonna have a really hard time debugging all that. So you're gonna end up pissed, you're gonna slam your laptop, and you're gonna go to bed, and this is gonna be a horrible experience that you could've prevented.

And you can prevent it by keeping your tests flat. And here's an example for you. This is a typical test, so we have a describe block that wraps the whole feature, it prepares some environment before all tests, then it has a sub-feature, for example, and it has its own setup, and then finally, the test. Even at this simple example, notice how many things we need to keep in mind, just to understand what this single test needs. So why not just put it into the test itself and drop the describe blocks altogether? And I hear you, it's gonna be pretty confusing and repetitive at first, but in time, you will grow to love this, because the benefits this gives are just incredible. It's declarative, it's explicit and you understand what each test needs from the test, from reading the test. And then of course, you can create and reuse test utilities to abstract commonly used logic. For example, if in this test we fill in a signing form and we do this very often to test the signing feature, well, why not abstract it into helper utility and call it sign in? And notice how immediately this reads much better, it reads like the intention, we want to sign in with this credentials. It doesn't matter what are the form selectors, what are the ideas and classes, it doesn't matter, the intention is to sign in and then do some expectations.

And of course, one of the most overlooked feature or like approaches is that you can split tests, you don't have to stuff all the tests in a single test file. So if you have a complicated feature like this sign in and it has different providers like email and GitHub, well, put them into separate test files and it's going to give you great readability and discoverability for the price of zero. And then when you need to add more logic and more tests, just add new test files and that's it. The same stands true when deleting features, because just as good code, good test is the one you can easily delete. It's the test that doesn't introduce a lot of implicit dependencies and all sorts of magic in the setup, which makes it really hard to remove.

So to summarize, tackling complexity in tests, it's very important to use the test phases properly and to do the most heavy lifting on the setup phase. And then of course reduce repetition in action phase. It's really crucial to express intentions using helper functions like the signing function I just showed you to help your test read like specification instead of a bunch of implementation details. It's really good to keep test structure flat so perhaps putting everything a single test needs in a single test block and of course use simple explicit assertions so you don't have to compute a lot of things in your head to understand what the test does. And when it comes to complex features, well you can also split them on the file system level and gain this great discoverability and great maintenance over time as your product develops. Of course there's much more to complexity, but that's all I have for today, so make sure to follow me on Twitter if you liked this talk and share with me some of your experiences with how you dealt with complexity in tests in the past.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Remix Conf Europe 2022Remix Conf Europe 2022
23 min
Scaling Up with Remix and Micro Frontends
Do you have a large product built by many teams? Are you struggling to release often? Did your frontend turn into a massive unmaintainable monolith? If, like me, you’ve answered yes to any of those questions, this talk is for you! I’ll show you exactly how you can build a micro frontend architecture with Remix to solve those challenges.
TestJS Summit 2021TestJS Summit 2021
33 min
Network Requests with Cypress
Whether you're testing your UI or API, Cypress gives you all the tools needed to work with and manage network requests. This intermediate-level task demonstrates how to use the cy.request and cy.intercept commands to execute, spy on, and stub network requests while testing your application in the browser. Learn how the commands work as well as use cases for each, including best practices for testing and mocking your network requests.
TestJS Summit 2021TestJS Summit 2021
38 min
Testing Pyramid Makes Little Sense, What We Can Use Instead
Featured Video
The testing pyramid - the canonical shape of tests that defined what types of tests we need to write to make sure the app works - is ... obsolete. In this presentation, Roman Sandler and Gleb Bahmutov argue what the testing shape works better for today's web applications.
Remix Conf Europe 2022Remix Conf Europe 2022
37 min
Full Stack Components
Remix is a web framework that gives you the simple mental model of a Multi-Page App (MPA) but the power and capabilities of a Single-Page App (SPA). One of the big challenges of SPAs is network management resulting in a great deal of indirection and buggy code. This is especially noticeable in application state which Remix completely eliminates, but it's also an issue in individual components that communicate with a single-purpose backend endpoint (like a combobox search for example).
In this talk, Kent will demonstrate how Remix enables you to build complex UI components that are connected to a backend in the simplest and most powerful way you've ever seen. Leaving you time to chill with your family or whatever else you do for fun.
JSNation Live 2021JSNation Live 2021
29 min
Making JavaScript on WebAssembly Fast
JavaScript in the browser runs many times faster than it did two decades ago. And that happened because the browser vendors spent that time working on intensive performance optimizations in their JavaScript engines.Because of this optimization work, JavaScript is now running in many places besides the browser. But there are still some environments where the JS engines can’t apply those optimizations in the right way to make things fast.We’re working to solve this, beginning a whole new wave of JavaScript optimization work. We’re improving JavaScript performance for entirely different environments, where different rules apply. And this is possible because of WebAssembly. In this talk, I'll explain how this all works and what's coming next.
React Summit 2023React Summit 2023
24 min
Debugging JS
As developers, we spend much of our time debugging apps - often code we didn't even write. Sadly, few developers have ever been taught how to approach debugging - it's something most of us learn through painful experience.  The good news is you _can_ learn how to debug effectively, and there's several key techniques and tools you can use for debugging JS and React apps.

Workshops on related topic

React Summit 2023React Summit 2023
151 min
Designing Effective Tests With React Testing Library
Featured Workshop
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents- The different kinds of React application tests, and where component tests fit in- A mental model for thinking about the inputs and outputs of the components you test- Options for selecting DOM elements to verify and interact with them- The value of mocks and why they shouldn’t be avoided- The challenges with asynchrony in RTL tests and how to handle them
Prerequisites- Familiarity with building applications with React- Basic experience writing automated tests with Jest or another unit testing framework- You do not need any experience with React Testing Library- Machine setup: Node LTS, Yarn
TestJS Summit 2022TestJS Summit 2022
146 min
How to Start With Cypress
Featured WorkshopFree
The web has evolved. Finally, testing has also. Cypress is a modern testing tool that answers the testing needs of modern web applications. It has been gaining a lot of traction in the last couple of years, gaining worldwide popularity. If you have been waiting to learn Cypress, wait no more! Filip Hric will guide you through the first steps on how to start using Cypress and set up a project on your own. The good news is, learning Cypress is incredibly easy. You'll write your first test in no time, and then you'll discover how to write a full end-to-end test for a modern web application. You'll learn the core concepts like retry-ability. Discover how to work and interact with your application and learn how to combine API and UI tests. Throughout this whole workshop, we will write code and do practical exercises. You will leave with a hands-on experience that you can translate to your own project.
React Summit 2022React Summit 2022
117 min
Detox 101: How to write stable end-to-end tests for your React Native application
Compared to unit testing, end-to-end testing aims to interact with your application just like a real user. And as we all know it can be pretty challenging. Especially when we talk about Mobile applications.
Tests rely on many conditions and are considered to be slow and flaky. On the other hand - end-to-end tests can give the greatest confidence that your app is working. And if done right - can become an amazing tool for boosting developer velocity.
Detox is a gray-box end-to-end testing framework for mobile apps. Developed by Wix to solve the problem of slowness and flakiness and used by React Native itself as its E2E testing tool.
Join me on this workshop to learn how to make your mobile end-to-end tests with Detox rock.
Prerequisites- iOS/Android: MacOS Catalina or newer- Android only: Linux- Install before the workshop
React Day Berlin 2022React Day Berlin 2022
86 min
Using CodeMirror to Build a JavaScript Editor with Linting and AutoComplete
Using a library might seem easy at first glance, but how do you choose the right library? How do you upgrade an existing one? And how do you wade through the documentation to find what you want?
In this workshop, we’ll discuss all these finer points while going through a general example of building a code editor using CodeMirror in React. All while sharing some of the nuances our team learned about using this library and some problems we encountered.
TestJS Summit 2023TestJS Summit 2023
48 min
API Testing with Postman Workshop
In the ever-evolving landscape of software development, ensuring the reliability and functionality of APIs has become paramount. "API Testing with Postman" is a comprehensive workshop designed to equip participants with the knowledge and skills needed to excel in API testing using Postman, a powerful tool widely adopted by professionals in the field. This workshop delves into the fundamentals of API testing, progresses to advanced testing techniques, and explores automation, performance testing, and multi-protocol support, providing attendees with a holistic understanding of API testing with Postman.
1. Welcome to Postman- Explaining the Postman User Interface (UI)2. Workspace and Collections Collaboration- Understanding Workspaces and their role in collaboration- Exploring the concept of Collections for organizing and executing API requests3. Introduction to API Testing- Covering the basics of API testing and its significance4. Variable Management- Managing environment, global, and collection variables- Utilizing scripting snippets for dynamic data5. Building Testing Workflows- Creating effective testing workflows for comprehensive testing- Utilizing the Collection Runner for test execution- Introduction to Postbot for automated testing6. Advanced Testing- Contract Testing for ensuring API contracts- Using Mock Servers for effective testing- Maximizing productivity with Collection/Workspace templates- Integration Testing and Regression Testing strategies7. Automation with Postman- Leveraging the Postman CLI for automation- Scheduled Runs for regular testing- Integrating Postman into CI/CD pipelines8. Performance Testing- Demonstrating performance testing capabilities (showing the desktop client)- Synchronizing tests with VS Code for streamlined development9. Exploring Advanced Features - Working with Multiple Protocols: GraphQL, gRPC, and more
Join us for this workshop to unlock the full potential of Postman for API testing, streamline your testing processes, and enhance the quality and reliability of your software. Whether you're a beginner or an experienced tester, this workshop will equip you with the skills needed to excel in API testing with Postman.
TestJS Summit - January, 2021TestJS Summit - January, 2021
173 min
Testing Web Applications Using Cypress
This workshop will teach you the basics of writing useful end-to-end tests using Cypress Test Runner.
We will cover writing tests, covering every application feature, structuring tests, intercepting network requests, and setting up the backend data.
Anyone who knows JavaScript programming language and has NPM installed would be able to follow along.