Measure and Improve Frontend Performance by Using Test Automation

Rate this content

Automated performance testing can help detect the harmful effects of code changes on application performance. Learn how to use tools like Lighthouse and Web Core Vitals in your CI and set performance thresholds to maintain optimal frontend performance in this session.

22 min
11 Dec, 2023

AI Generated Video Summary

The Talk focuses on the importance of testing and gathering information for building good applications. It highlights the use of test automation for performance monitoring and logging for performance measurement. The Talk also discusses the impact of performance on user engagement and search engine rankings. It emphasizes the use of Cypress plugins for monitoring performance metrics and setting thresholds for tests. Overall, the Talk emphasizes the value of test automation tools in providing valuable information at a low cost.

1. Introduction to Testing and Information

Short description:

Hello, everyone at TestJS Summit this time. I'm so glad to be here again. Today, I'm coming home with an interesting discovery on performance. I'm passionate about testing and believe that information is king. Gathering information is important for building good applications. Make it work, make it right, make it fast - the order of building good applications.

Hello, everyone at TestJS Summit this time. I'm so glad to be here again because the TestJS summit was one of the first conferences I was honored to be a speaker at. It's just like coming home today.

Today, I'm actually coming home here with a little interesting discovery I want to showcase to you when it gets to my testing experience. If you might have guessed already, it's on performance. Well, before that, real quick, my name is Ramona. I'm working as a Developer Advocate at Offs Zero. Apart from that, I'm a Google Developer Expert in web technology and a Cypress Ambassador. As you might have guessed, I think I'm, yeah, at least I'm passionate when it comes to testing. Yeah, it shouldn't be a surprise that I'm here to talk about testing today.

So well, especially when it comes to testing, there are some things which are really important. And yeah, this little thing, Content is King, which I hope I have enough for this 20 minutes. It was done by Steve Bommer, I guess. He's doing lots of quotes, which are memorable. But well, let's change this quote a little bit. Because, if Content is King, Information is King like, doubled or even more. And it's not just because I like to play video games and this is the title of a World of Warcraft quest. But I, well, find interesting. At least I like those little sidequests where, for example in this case, a goblin vendor named Goodskitch wants to have some Krukal skin in exchange for information. So, it's not only in video games that information is really important. Gathering information, having a good grasp on what happens inside of your application, is really important. Not only in gaming, but also when it comes to testing.

So, well, I guess this should be clear for most of the time. But, well, why do we need to have information on our application? Because we want to build good applications, right? So, you might have stumbled upon this quote by Comeback, which says, make it work, make it right, make it fast, in that particular order. So, just to dive into it a little bit, make it work is self-explanatory. So try to solve a problem at hand, which might build a good application. So, well, just make it work when it comes to your requirements. If you are able to do that, do the second thing, make it maintainable, make it right. So, try to mind non-functional requirements. Mind clean code, for example, to make it not only work, but work the right way and work the way that you still want to work with your application, let's say, one year from now.

2. Using Test Automation for Performance Monitoring

Short description:

And last but not least, make it perform, make it fast, make it that the user is not annoyed by using your application. We can use automation to support us in all of those three points, even performance. Test automation can be used for monitoring performance and gathering information. Logging is the art of keeping a log or a list of events that occur in your computer system, such as problems, errors, or just information in the current operations.

And last but not least, make it perform, make it fast, make it that the user is not annoyed by using your application. And yes, of course, the first point, make it work is pretty easily achieved with testing. So, if you test your application, you know that it's working as intended. The second one, make it right can be supported by tooling, for example, if we had good tests, making it easier to write good code. And I don't want to start with all the linches, all the static analysis tools you can run, like phpStan, ESlearnStyle and whatever. But, well, we have good support when it comes to first, like, make it work, make it right. But what is with the third point, make it fast and perform and basically make it feel flawless? Well, make your application perform seems really daunting, because as I said, you don't have that much tool support yet. And well, even the point, the steps of speeding up your application code wise is really difficult, because you could ask, like, who has time for that, who has the power to do that? And may it be because of pressure by clients, but especially head space inside of yourself. And do we get the time for improving the performance or maintaining it? Because even getting all the information you need on your application to monitor performance, doing all those measures locally takes lots of time. And all of this together seems really daunting. So what if I told you some interesting point that we don't need to feel lost. We don't need to feel so drained or scared. What if I told you that we can use automation to support us in all of those three points? Even performance. Not only working, not only making it right, making it perform. What if I told you you can use test automation for that? Not only trying to learn GitHub Actions to implement some tools to help us to monitor performance, but using everything we already have. Test automation we already have for our project. Well, I know, I know, it's a little like misusing testing, right? So well, it's still good because you can use your knowledge you already have for it. You don't need to learn new tools. You can use your well known familiar CI. For example, if you don't want to use GitHub Actions, which can give it to you, or if a standardized GitHub Action is not enough to fulfill your needs. So, in my mind I think it's interesting to use end to end testing for measuring performance because the AppStack application stack is completely there, and existent. So we can do more of it, than only when it comes to unit tests. And we are close to the user who would be annoyed by performance problems, right? But, if we want to think about using test automation for monitoring performance and gathering, monitoring, and collecting information on our performance, let's take a step back and take a look how we measure or collect information on our test automation to begin with. Well, as this talk is called Measuring, let's start with the defaults to shed some light on it.

Many think about the most interesting or most normal way of taking a look at information from your test automation pipelines. For example, if you want to take a look why something fails, you will take a look at the logs. Logging is actually the art of keeping a log or a list of events that occur in your computer system, such as problems, errors, or just information in the current operations. It might be testing, for example, in this regard. It logs everything what happens inside of your tests. It looks like that.

3. Using Logging for Performance Measurement

Short description:

CRI logging is a vital part for enabling you to measure things. We need to be able to notice the problems we have and make it fast. There are ways to support you in having logging as easy and accessible as possible. Let's see how we can use logging to measure performance values. Performance in web automation depends on factors like page load time, responsiveness, and overall user experience. To achieve a high performing website, we can optimize by reducing file sizes and minimizing server requests.

CRI logging is visible inside of your CRI. Even if the presentation is differing sometimes, it's handled similar in many frameworks, no matter if it's Cypress, no matter if it's PlayWrite. So, you can take a look at the test results and of course to the errors, if there are any. And some frameworks give you a little more insights on it.

For example, here the test runner of Cypress and PlayWrite has a similar one. But all of those frameworks make it clear what happens inside of your test. Well, you could wonder now, do I tell you that right now? Why should I bother because it's a standard feature, right? You're right. But there's something you need to keep in your mind, because logging is a vital part for enabling you to measure things. And if you cannot measure something, you can't improve it, right? Because you don't have a messenger to tell you that something's wrong. So, yeah, we need to remember the make it part. This is crucial. We need to be able to notice the problems we have. So especially the point of making it fast is important in our regard. So we are dependent on metrics. So we need to let the metrics show as a base of comparison. And before you worry right now, you don't need to always handle those daunting log outputs.

There are a couple of ways to support you to have this logging as easy and as accessible as possible. For example, if you want to use a custom reporter to showcase it a little better, which could be more awesome, but there are many more which you could take a look at. Well, there are plenty of plugins who extend the log output inside of your CI. Maybe Cypress plugin, in my case, where I will add the name later on because I seemingly have forgotten it. So you could think about printing of the DevTool console output or request to make it even easier. Well, so far so good. These are logging defaults. So let's see how we can use them in detail to misuse test automation to measure performance values. And before we do that, of course, this talk is about performance, but I guess we should have a common understanding of what performance is in this regard.

So performance in web automation, we first speed and efficiency with which a website or web application functions and delivers content to the user. And it can depend on various factors like page load time, responsiveness and overall user experience. Basically everything which might let you feel triggered or let's say it like annoyed or something which doesn't feel like a flawless user experience that you cannot do your job on. And our dream of a high performing website is that it looks quickly, it allows users to access information without delays or interruptions. So it's just flawless, right? To get to that point, we can do some optimizing which could involve optimizing reducing file sizes, minimizing the number of requests made to the server.

4. Performance Testing and Monitoring

Short description:

All of those little things which add up pretty quickly. And yeah, if we don't take a look at this, we will have higher bounce rates, lower user engagement and a negative impact on Search Engine rankings. So when it comes to performance testing, there are a couple of ways on how you could use this. It's the front-end client side here, because we have front-end test automation here by utilising Javascript. Monitoring performance in the front-end makes a lot of sense. The most common tool everyone uses to track front-end performance is Lighthouse. You could use the Cypress Audit for Cypress tasks, there's a Playwright Alternative too. Everything in between 0 and 49 is bad and probably will influence your ranking. For our part, the performance, there are some interesting metrics named here like first contentful paint and largest contentful paint which are really interesting for us. The three core web vitals metrics are first, the largest content for paint, the first input delay, and the cumulative layout shift. And there is a Cypress Plugin 2, which you could use to monitor the core web vitals through our test automation.

All of those little things which add up pretty quickly. And yeah, if we don't take a look at this, we will have higher bounce rates, lower user engagement and a negative impact on Search Engine rankings.

So when it comes to performance testing, which is basically the type of testing conducted to evaluate the speed, the responsiveness, stability and scalability of application or website, there are a couple of ways on how you could use this. So when I'm thinking about stuff like load testing, where you have the normally expected user loads to determine its response time, but also stress testing, spike testing, when it comes to load spikes on how to handle it in your application, even stress testing, to try to push your website beyond its normal capacity, there are two aspects of performance testing you could utilise to get a bigger picture of your application's behaviour.

It could be back advice, of course, so the most traditional tools, I don't cover those, at all cost, but I want to focus on another thing, it's the front-end client side here, because we have front-end test automation here by utilising Javascript, but not only that, many people don't remember that most of the load time of a web page is spent in the front-end, as Marie says, I hope you have take a look at her talk, because she's awesome, but monitoring performance in the front-end makes a lot of sense, if most people will notice it in the front-end, and if you think about, okay, how can I monitor front-end audits, front-end metrics, I guess we share one similar thought, and basically the first step we take a look at when it comes to those topics, well, it's evaluating Lighthouse, so Lighthouse is basically the most common tool everyone uses to track front-end performance, and many people use it because it's pretty easy because you can access it via Chrome DevTools.

And I guess when it comes to debugging locally, many of you guys maybe already tried it out, but were you aware that you could automate this? You could use a GitHub Action which is I guess even predefined, but you could use your end-to-end test to keep track of those too. There's a plugin which is called Cypress Audit which provides Lighthouse audits but also Pally audits, but I don't want to focus on accessibility right now because it's I guess enough for the slot here. So Lighthouse it is. So you could use the Cypress Audit for Cypress tasks, there's a Playwright Alternative too, and there you take a look at the typical Lighthouse thresholds. The categories what we use for Lighthouse and provide the score between 0 and 100 in the realm of performance, which is important for us. Accessibility, best practices, SEO and PWA.

And when it comes to what's good and what's not, we could zoom on the performance audit here and we see that everything in between 0 and 49 is bad and probably will influence your ranking. Then everything in between 50 and 89 is something with some some possibilities of improvement and everything above 19 is great and you can see that here right now. And for our part, the performance, there are some interesting metrics named here like first contentful paint and largest contentful paint which are really interesting for us. We will take a closer look because these are the ones we want to take a closer look and we want to act on those. Okay, those two terms were taken from Google's web vitals which are a set of specific metrics that measure the user experience of a website. They focus on three critical aspects of the web performance loading, like how fast something is, interactivity, how fast the application reacts to our input and visual stability. So maybe you already happen to have a mobile page. You are scrolling through it, you want to click something, but it jumps. Yeah, this is something we need to take a look at. And yeah, as said those metrics are by Google and they have implications on the website. So the three core web vitals metrics are first, the largest content for paint, which measures how quickly the main content of a web page looks. It should occur within a 2.5 second or when the page first starts loading. The second one is the first input delay. It measures the time it takes for a web page to become interactive. Should be less than a hundred milliseconds. Last but not least, the cumulative layout shift, which measures the visual stability of a web page by tracking unexpected layout shifts of elements. And it should have a score of less than 0.1. And there are many more metrics, of course, but we will cover the core ones, the core web vitals here. And there is a Cypress Plugin 2, which you could use to monitor the core web vitals through our test automation.

5. Using Cypress Plugins for Performance Monitoring

Short description:

Use Cypress plugins to monitor performance metrics and set thresholds for your tests. Tools like Lighthouse and Cypress web writers can help you monitor website performance and detect failures. Treat your end testing job pipeline as a detective that checks metrics every night. Remember to measure performance, focus on the front-end, and use test automation to capture performance metrics. Test automation tools can be more than just testing tools. They provide valuable information at a low cost.

So use this QR code to find it. And let me showcase it a little bit. So you could install it via npm or yarn, or your favorite packet manager. And then you can import the commands so Cypress knows them. Again, the typical Cypress installation workflow when it comes to plugins. As we see it here.

Afterwards, you could basically start using it inside of your tests. So I will import it in the commands.js. And then I will use an existing blank test to include the cypress-writers command. And use some web-writers config, which is the URL I want to take a look at. And apart from that, it's just, I want to use the normal thresholds. And as you see it here, something errors because a threshold has been crossed. So the threshold we saw in the slide before. And yeah, the test is failing if we don't need those. And well, if you see such an example with some metrics in it, you could even use other thresholds if you want to. For example, I added the first content for page, which is the first displayed element, how long it loads. Well, the time to first byte, the time in between the request for resource. And when the first byte of a response begins to arrive, which are useful metrics too. And if you take a look on how it is locked, you can see it here in the test runner, you will see, yeah, the fail visible inside of the log. This is the log. And of course, the result is there too. And you can be sure that your test will fail if it doesn't match the thresholds. Okay.

So, these tools like Lighthouse and the Cypress web writers will help you monitor your website's performance and make your test fail, which is really great. So, you could think of a pipeline in your end testing job like a little detective, who goes there every night and takes a look at the metrics of your application. And fail visible performance metrics aren't hitting the thresholds. You can see that through login. And yeah, you have your own Sherlock Cypress or Sherlock Playwright if you want to. So, it provides you with lots of information with not that much of a cost. Great, right? So, if you want to remember four things from my talk, which are really important when it comes to monitoring and improving front-end performance by utilizing test automation is always measure performance to improve it because otherwise you have no possibility to improve it. Performance issues are often noticed inside front-end, so focus here first to measure. You can use your normal U2E test automation to capture most of the performance metrics and light holes and especially cobalt vitrates can be used easily as cypress clutters. So, I said you can use test automation tools for more than just testing. I hope they can feel like an assistant to you, so feel free to use them and hopefully they are helping you a lot. So, what else to say then? Thank you for your time and for listening to me.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

React Advanced Conference 2022React Advanced Conference 2022
25 min
A Guide to React Rendering Behavior
React is a library for "rendering" UI from components, but many users find themselves confused about how React rendering actually works. What do terms like "rendering", "reconciliation", "Fibers", and "committing" actually mean? When do renders happen? How does Context affect rendering, and how do libraries like Redux cause updates? In this talk, we'll clear up the confusion and provide a solid foundation for understanding when, why, and how React renders. We'll look at: - What "rendering" actually is - How React queues renders and the standard rendering behavior - How keys and component types are used in rendering - Techniques for optimizing render performance - How context usage affects rendering behavior| - How external libraries tie into React rendering
React Summit 2023React Summit 2023
32 min
Speeding Up Your React App With Less JavaScript
Too much JavaScript is getting you down? New frameworks promising no JavaScript look interesting, but you have an existing React application to maintain. What if Qwik React is your answer for faster applications startup and better user experience? Qwik React allows you to easily turn your React application into a collection of islands, which can be SSRed and delayed hydrated, and in some instances, hydration skipped altogether. And all of this in an incremental way without a rewrite.
TestJS Summit 2021TestJS Summit 2021
33 min
Network Requests with Cypress
Whether you're testing your UI or API, Cypress gives you all the tools needed to work with and manage network requests. This intermediate-level task demonstrates how to use the cy.request and cy.intercept commands to execute, spy on, and stub network requests while testing your application in the browser. Learn how the commands work as well as use cases for each, including best practices for testing and mocking your network requests.
React Summit 2023React Summit 2023
23 min
React Concurrency, Explained
React 18! Concurrent features! You might’ve already tried the new APIs like useTransition, or you might’ve just heard of them. But do you know how React 18 achieves the performance wins it brings with itself? In this talk, let’s peek under the hood of React 18’s performance features: - How React 18 lowers the time your page stays frozen (aka TBT) - What exactly happens in the main thread when you run useTransition() - What’s the catch with the improvements (there’s no free cake!), and why Vue.js and Preact straight refused to ship anything similar
TestJS Summit 2021TestJS Summit 2021
38 min
Testing Pyramid Makes Little Sense, What We Can Use Instead
Featured Video
The testing pyramid - the canonical shape of tests that defined what types of tests we need to write to make sure the app works - is ... obsolete. In this presentation, Roman Sandler and Gleb Bahmutov argue what the testing shape works better for today's web applications.
JSNation 2022JSNation 2022
21 min
The Future of Performance Tooling
Our understanding of performance & user-experience has heavily evolved over the years. Web Developer Tooling needs to similarly evolve to make sure it is user-centric, actionable and contextual where modern experiences are concerned. In this talk, Addy will walk you through Chrome and others have been thinking about this problem and what updates they've been making to performance tools to lower the friction for building great experiences on the web.

Workshops on related topic

React Summit 2023React Summit 2023
170 min
React Performance Debugging Masterclass
Featured WorkshopFree
Ivan’s first attempts at performance debugging were chaotic. He would see a slow interaction, try a random optimization, see that it didn't help, and keep trying other optimizations until he found the right one (or gave up).
Back then, Ivan didn’t know how to use performance devtools well. He would do a recording in Chrome DevTools or React Profiler, poke around it, try clicking random things, and then close it in frustration a few minutes later. Now, Ivan knows exactly where and what to look for. And in this workshop, Ivan will teach you that too.
Here’s how this is going to work. We’ll take a slow app → debug it (using tools like Chrome DevTools, React Profiler, and why-did-you-render) → pinpoint the bottleneck → and then repeat, several times more. We won’t talk about the solutions (in 90% of the cases, it’s just the ol’ regular useMemo() or memo()). But we’ll talk about everything that comes before – and learn how to analyze any React performance problem, step by step.
(Note: This workshop is best suited for engineers who are already familiar with how useMemo() and memo() work – but want to get better at using the performance tools around React. Also, we’ll be covering interaction performance, not load speed, so you won’t hear a word about Lighthouse 🤐)
React Summit 2023React Summit 2023
151 min
Designing Effective Tests With React Testing Library
Featured Workshop
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents- The different kinds of React application tests, and where component tests fit in- A mental model for thinking about the inputs and outputs of the components you test- Options for selecting DOM elements to verify and interact with them- The value of mocks and why they shouldn’t be avoided- The challenges with asynchrony in RTL tests and how to handle them
Prerequisites- Familiarity with building applications with React- Basic experience writing automated tests with Jest or another unit testing framework- You do not need any experience with React Testing Library- Machine setup: Node LTS, Yarn
JSNation 2023JSNation 2023
170 min
Building WebApps That Light Up the Internet with QwikCity
Featured WorkshopFree
Building instant-on web applications at scale have been elusive. Real-world sites need tracking, analytics, and complex user interfaces and interactions. We always start with the best intentions but end up with a less-than-ideal site.
QwikCity is a new meta-framework that allows you to build large-scale applications with constant startup-up performance. We will look at how to build a QwikCity application and what makes it unique. The workshop will show you how to set up a QwikCitp project. How routing works with layout. The demo application will fetch data and present it to the user in an editable form. And finally, how one can use authentication. All of the basic parts for any large-scale applications.
Along the way, we will also look at what makes Qwik unique, and how resumability enables constant startup performance no matter the application complexity.
TestJS Summit 2022TestJS Summit 2022
146 min
How to Start With Cypress
Featured WorkshopFree
The web has evolved. Finally, testing has also. Cypress is a modern testing tool that answers the testing needs of modern web applications. It has been gaining a lot of traction in the last couple of years, gaining worldwide popularity. If you have been waiting to learn Cypress, wait no more! Filip Hric will guide you through the first steps on how to start using Cypress and set up a project on your own. The good news is, learning Cypress is incredibly easy. You'll write your first test in no time, and then you'll discover how to write a full end-to-end test for a modern web application. You'll learn the core concepts like retry-ability. Discover how to work and interact with your application and learn how to combine API and UI tests. Throughout this whole workshop, we will write code and do practical exercises. You will leave with a hands-on experience that you can translate to your own project.
React Day Berlin 2022React Day Berlin 2022
53 min
Next.js 13: Data Fetching Strategies
- Introduction- Prerequisites for the workshop- Fetching strategies: fundamentals- Fetching strategies – hands-on: fetch API, cache (static VS dynamic), revalidate, suspense (parallel data fetching)- Test your build and serve it on Vercel- Future: Server components VS Client components- Workshop easter egg (unrelated to the topic, calling out accessibility)- Wrapping up
React Summit 2022React Summit 2022
117 min
Detox 101: How to write stable end-to-end tests for your React Native application
Compared to unit testing, end-to-end testing aims to interact with your application just like a real user. And as we all know it can be pretty challenging. Especially when we talk about Mobile applications.
Tests rely on many conditions and are considered to be slow and flaky. On the other hand - end-to-end tests can give the greatest confidence that your app is working. And if done right - can become an amazing tool for boosting developer velocity.
Detox is a gray-box end-to-end testing framework for mobile apps. Developed by Wix to solve the problem of slowness and flakiness and used by React Native itself as its E2E testing tool.
Join me on this workshop to learn how to make your mobile end-to-end tests with Detox rock.
Prerequisites- iOS/Android: MacOS Catalina or newer- Android only: Linux- Install before the workshop