The Life-Changing Magic of Tidying Up your Test Warnings

Rate this content
Bookmark

Even though we write tests for our web applications, the reality is that bugs still happen. Fortunately, many of these are easily preventable paying more attention to the warnings from our apps. However, it's often so easy to put them under the rug and never come back until we find a bug in production, which leads to hundreds if now thousands of warnings appearing in our test output. This talk is about how to prevent this situation and how to get out of it.

8 min
15 Jun, 2021

Comments

Sign in or register to post your comment.

AI Generated Video Summary

Today's Talk focuses on preventing test warnings in software development. Test warnings are often ignored and can lead to bugs, performance issues, and security concerns. The speaker introduces a library called jsreporter log validator that automates the process of adding rules to prevent new warnings and fixing existing ones. The library provides a summary of expected behavior, failures, and actions to take. Overall, the Talk emphasizes the importance of paying attention to test warnings and using automation to improve developer experience and prevent issues in large and legacy applications.

1. Introduction to Test Warnings

Short description:

Today we're going to be talking about preventing test warnings, with two goals in mind: preventing bugs and improving developer experience. Test warnings are messages created by developers to avoid bugs, performance issues, security concerns, and more. Warnings tend to accumulate because they're easy to ignore, don't make CI fail, and are often not a priority. Ignoring warnings can have consequences, as I'll demonstrate with a small application example.

Hello, everyone, and thank you for joining to the life-changing magic of tiding up your test warnings. Today we're going to be talking about preventing test warnings, with two goals in mind. The first one is going to be to prevent bugs. This is the most important one, and the second one is to improve developer experience.

A little bit about myself. My name is Victor Cordova. I work at TravelPerk, a Barcelona start-up. We're building the world's best business travel platform. If you're interested, please feel free to join us.

All right. So let's start by asking what are test warnings for? Test warnings are essentially messages created by developers of third-party libraries or other technologies that give us clues about what to avoid. For example, we want to definitely avoid bugs. We want to avoid performance issues. We want to avoid security concerns, amongst many others. This is just a very small sample. We also have accessibility issues, deprecations, and so on.

Now, the thing about warnings is that they tend to accumulate with time, and it's worth it to ask why this is the case. The first one is because they're pretty easy to ignore. So essentially, test warnings are just texts being generated either in your local machine or on another server. So this text, by itself, doesn't do anything. The second reason is because they don't make your CI fail. As developers, we all know that we pay much more attention to this red color that pops up whenever something fails. And finally, because they usually are not a priority. We live in a complex world. We have product tasks, technical tasks, so warnings can easily go to the end of this list.

Now, it's important to ask why do we even care, honestly. I ask myself that. So what happens if we ignore warnings? I'm going to give you a very small sample of what can happen. This is a small application with a book inventory where we have the title, the registration date, and the condition of the book. So let's imagine I'm going to fill this right now.

2. Preventing Test Warnings

Short description:

I'm going to put fair, good, and terrible. Now, why is this concerning? Because this might very easily be your output in the test run. React will give you a warning that says, every element must have a unique key. That's why we need to pay attention to these warnings. This is the developer experience side of things. If you're trying to do TDD, if you're trying to debug an issue, nobody wants to see this. It's quite annoying. The developer experience is affected. It's a very common issue in large applications, legacy applications. But we're engineers. So let's use some automation. I created this very small library called jsreporter log validator. It allows you to add different rules to your warnings so that your team doesn't create new ones. You can add validations for certain patterns. Sometimes they have a dynamic part. You can put a maximum. You can also have a fail-safe for unavoidable warnings. We sometimes install third-party libraries that generate messages we don't want. But we cannot do anything about it at times. We also have an option to fail if an unknown warning is found.

I'm going to put fair, good, and terrible. So now I'm going to try to use this sorting functionality. And we'll see that everything but the condition is sorted.

Now, why is this concerning? Because this might very easily be your output in the test run. So everything is green, which is really not a reflection of what's happening. So React, which is just an example, will give you a warning that says, every element must have a unique key. That's why we need to pay attention to these warnings. This is the developer experience side of things. If you're trying to do TDD, if you're trying to debug an issue, nobody wants to see this. It's quite annoying. It's difficult to find important stuff. So the developer experience is affected.

Now, we can ask ourselves what do we do about it? It's a very common issue in large applications, legacy applications. And it feels sometimes like we really can't do anything. But we're engineers. So let's use some automation. So I created this very small library. It's called jsreporter log validator. And it allows you to add these different rules to your warnings so that your team doesn't create new ones. The first feature it has is that you can add validations for certain patterns. As you will see, it's not a single string for each one of the patterns. Sometimes they have a dynamic part. You can put a maximum. So you are basically saying, okay, we know we have this number of warnings of this type. But I don't want any more. The second one is that you can have a fail-safe for unavoidable warnings. We sometimes install third-party libraries that generate messages we don't want. But we cannot do anything about it at times. So we can just ignore it for now. We also have an option to fail if an unknown warning is found.

3. Using Automation to Fix Warnings

Short description:

We have a registry of allowed warnings that can be updated with new versions of libraries. When fixing warnings, you can ensure they no longer occur. The library provides a summary of expected behavior, failures, and actions to take. Feel free to experiment and provide feedback on this automation solution.

So again, we have this registry of allowed warnings. If something else comes up, let's say, you create a pull request updating the new version of a library, then you will know in your build that it has to fail. And if you want it to pass, then you will add it to this config, but at least you will be explicit. And finally, let's say you are actually starting to fix your warnings. You said, okay, this warning, you cannot change router. I don't want it anymore. And I want to fix it. So somebody fixes it. But you want this to disappear. You don't want to allow for anymore. So this last setting allows you to keep this up-to-date. This is an example of how it looks like. It will give you a nice summary of what's the expected behavior, what failed and what you can do about it. Just a small, a small caveat. This library is in early stages. So feel free to play around with it. And also to give some feedback. But this is just an example of how you can use automation to fix this issue.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

TestJS Summit 2021TestJS Summit 2021
33 min
Network Requests with Cypress
Whether you're testing your UI or API, Cypress gives you all the tools needed to work with and manage network requests. This intermediate-level task demonstrates how to use the cy.request and cy.intercept commands to execute, spy on, and stub network requests while testing your application in the browser. Learn how the commands work as well as use cases for each, including best practices for testing and mocking your network requests.
TestJS Summit 2021TestJS Summit 2021
38 min
Testing Pyramid Makes Little Sense, What We Can Use Instead
Featured Video
The testing pyramid - the canonical shape of tests that defined what types of tests we need to write to make sure the app works - is ... obsolete. In this presentation, Roman Sandler and Gleb Bahmutov argue what the testing shape works better for today's web applications.


TestJS Summit 2021TestJS Summit 2021
31 min
Test Effective Development
Developers want to sleep tight knowing they didn't break production. Companies want to be efficient in order to meet their customer needs faster and to gain competitive advantage sooner. We ALL want to be cost effective... or shall I say... TEST EFFECTIVE!
But how do we do that?
Are the "unit" and "integration" terminology serves us right?
Or is it time for a change? When should we use either strategy to maximize our "test effectiveness"?
In this talk I'll show you a brand new way to think about cost effective testing with new strategies and new testing terms!
It’s time to go DEEPER!


TestJS Summit 2022TestJS Summit 2022
27 min
Full-Circle Testing With Cypress
Cypress has taken the world by storm by brining an easy to use tool for end to end testing. It’s capabilities have proven to be be useful for creating stable tests for frontend applications. But end to end testing is just a small part of testing efforts. What about your API? What about your components? Well, in my talk I would like to show you how we can start with end-to-end tests, go deeper with component testing and then move up to testing our API, circ
TestJS Summit 2021TestJS Summit 2021
36 min
Effective Performance Testing to your Server with Autocannon
Performance testing expertise that is developed for a long time. In order to measure your server performance you need a tool that can efficiently simulate a lot of abilities and give you good measurements according your analysing criteria.
Autocannon NPM library gave me exactly that - that library is super easy to install and has a very simple API to work with. Within a really short amount of time you can start do performance testing to your application and get good measurements in development environment and in your performance labs, and generate complicated testing scenarios.
In this talk I will introduce Autocannon, explain how to efficiently analyse your server performance with it, and show how it helped me to understand complicated performance issues in my Node.js servers. At the end of this lecture, developers will be able to have the ability to integrate a fast and easy tool in order to measure your server performance.

Workshops on related topic

React Summit 2023React Summit 2023
151 min
Designing Effective Tests With React Testing Library
Featured Workshop
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents
- The different kinds of React application tests, and where component tests fit in
- A mental model for thinking about the inputs and outputs of the components you test
- Options for selecting DOM elements to verify and interact with them
- The value of mocks and why they shouldn’t be avoided
- The challenges with asynchrony in RTL tests and how to handle them
Prerequisites
- Familiarity with building applications with React
- Basic experience writing automated tests with Jest or another unit testing framework
- You do not need any experience with React Testing Library
- Machine setup: Node LTS, Yarn
TestJS Summit 2022TestJS Summit 2022
146 min
How to Start With Cypress
Featured WorkshopFree
The web has evolved. Finally, testing has also. Cypress is a modern testing tool that answers the testing needs of modern web applications. It has been gaining a lot of traction in the last couple of years, gaining worldwide popularity. If you have been waiting to learn Cypress, wait no more! Filip Hric will guide you through the first steps on how to start using Cypress and set up a project on your own. The good news is, learning Cypress is incredibly easy. You'll write your first test in no time, and then you'll discover how to write a full end-to-end test for a modern web application. You'll learn the core concepts like retry-ability. Discover how to work and interact with your application and learn how to combine API and UI tests. Throughout this whole workshop, we will write code and do practical exercises. You will leave with a hands-on experience that you can translate to your own project.
React Summit 2022React Summit 2022
117 min
Detox 101: How to write stable end-to-end tests for your React Native application
WorkshopFree
Compared to unit testing, end-to-end testing aims to interact with your application just like a real user. And as we all know it can be pretty challenging. Especially when we talk about Mobile applications.
Tests rely on many conditions and are considered to be slow and flaky. On the other hand - end-to-end tests can give the greatest confidence that your app is working. And if done right - can become an amazing tool for boosting developer velocity.
Detox is a gray-box end-to-end testing framework for mobile apps. Developed by Wix to solve the problem of slowness and flakiness and
used by React Native itself
as its E2E testing tool.
Join me on this workshop to learn how to make your mobile end-to-end tests with Detox rock.
Prerequisites
- iOS/Android: MacOS Catalina or newer
- Android only: Linux
-
Install before the workshop
TestJS Summit - January, 2021TestJS Summit - January, 2021
173 min
Testing Web Applications Using Cypress
WorkshopFree
This workshop will teach you the basics of writing useful end-to-end tests using Cypress Test Runner.
We will cover writing tests, covering every application feature, structuring tests, intercepting network requests, and setting up the backend data.
Anyone who knows JavaScript programming language and has NPM installed would be able to follow along.


TestJS Summit 2021TestJS Summit 2021
85 min
Automated accessibility testing with jest-axe and Lighthouse CI
Workshop
Do your automated tests include a11y checks? This workshop will cover how to get started with jest-axe to detect code-based accessibility violations, and Lighthouse CI to validate the accessibility of fully rendered pages. No amount of automated tests can replace manual accessibility testing, but these checks will make sure that your manual testers aren't doing more work than they need to.


React Advanced Conference 2023React Advanced Conference 2023
159 min
Effective Detox Testing
Workshop
So you’ve gotten Detox set up to test your React Native application. Good work! But you aren’t done yet: there are still a lot of questions you need to answer. How many tests do you write? When and where do you run them? How do you ensure there is test data available? What do you do about parts of your app that use mobile APIs that are difficult to automate? You could sink a lot of effort into these things—is the payoff worth it?
In this three-hour workshop we’ll address these questions by discussing how to integrate Detox into your development workflow. You’ll walk away with the skills and information you need to make Detox testing a natural and productive part of day-to-day development.
Table of contents:
- Deciding what to test with Detox vs React Native Testing Library vs manual testing
- Setting up a fake API layer for testing
- Getting Detox running on CI on GitHub Actions for free
- Deciding how much of your app to test with Detox: a sliding scale
- Fitting Detox into you local development workflow
Prerequisites
- Familiarity with building applications with React Native
- Basic experience with Detox
- Machine setup: a working React Native CLI development environment including either Xcode or Android Studio