A Medley of Frontend and Backend Performance Testing

♪♪♪ Hey, everyone. My name's Marie. And welcome to my talk called A Medley of Front-end and Back-end Performance Testing. Before I start with my talk, I want to tell you a story first. This story is about Overcooked. I've been playing this game with my 5-year-old daughter. And if you're not so familiar with Overcooked, it's a cooperative game where you have to pass different and unusual kitchen layouts and serve as much food as possible to customers.

When the game starts, it's still pretty normal. Orders are flowing in nicely, and you're getting tips because you're getting sushi orders served on time. As the game gets harder and you get more orders than expected, the kitchen is now overwhelmed. And without proper coordination and teamwork in place, the kitchen is now on fire. You're also not getting tips anymore, and you've got hangry customers waiting impatiently for their food. Because the kitchen can't keep up with the overflowing orders from customers, the whole kitchen is now on fire. Of course, this is very dramatic, but ultimately, you get the picture. The customers are very unhappy, you're getting negative tips, and the kitchen is such a mess that you can't even cook a single potato.

Going back to my topic of performance testing, imagine you are trying to buy some items during Black Friday or Cyber Monday sales. You found an item that you really like, but suddenly, the website that you were using has crashed. It can't keep up with the overwhelming requests from different users simultaneously. This is a very common phenomenon during Black Friday sales. You can see in this example graph as well that during the peak times of Black Friday sales, the response times are significantly higher as opposed to normal periods. This has then resulted to response times errors that can break your website. Most of the time, the response from companies is to buy more servers, thinking that this will fix their performance problems, but this could end up costing you more money. A better investment is if you understand, test, monitor and make performance improvements to your internal application.

Now let's get to the more serious part of this talk. In order for us to make sure that our users have a positive user experience, we need to do performance testing. Performance testing is the practice of measuring and evaluating how your system responds under certain conditions. When you think of performance testing, we are concerned about the speed, reliability and stability of the application that we are testing.

With performance testing, there is often a misconception that it's all about load testing. Performance testing is the umbrella term for any type of performance test while load testing is just one type of performance testing. In a nutshell, load testing checks how your application is behaving if it's been exposed to a heavy number of concurrent virtual users, sending multiple requests at a given time. Within load testing, there's also different variations such as stress testing, soak testing, or spike testing.

Performance testing is typically divided into two areas. We have the front-end or client-side performance, which is aimed at testing how fast a user can see the web responses instantly. It is concerned with the end user experience of an application, which usually involves a browser. Front-end performance testing has metrics that are distinct from back-end performance testing. Example metrics could be, how long did it take for the browser to render the entire page or how long did it take for the page to be fully interactive? Then on the other hand, we have back-end or server-side performance testing, which is aimed at ensuring that when multiple requests are sent from different users, simultaneously, your back-end should be able to handle the load accordingly. Example metrics could be, how long did it take for a response to come back from a specific request or how many failed requests did we encounter?

So, as you can see, performance testing is not just about load testing. With different types of performance testing, you might wonder if there is a priority as to which one is more important. The answer, as with everything, is that it depends, it always is. If we revisit the golden rule of web performance, it states that 80 to 90% of the load time of a web page or application is spent in the front end while 10 to 20% is spent in the back-end. You can see in this image, which I took from Steve Souder's blog, that the average front-end time is significantly higher as opposed to the back-end timings. If we are following this golden rule and you want to make some performance improvements, it's always a great idea to start on the front end and make small recommendations to your team. Performance testing on the front end is also much closer to our users' experience. However, the golden rule of web performance is not always necessarily accurate. If you have a lot of traffic arriving at your website, the front-end response time can remain roughly similar. But once your back-end struggles with the increased concurrency, the back-end time will grow exponentially. Front-end performance testing is executed on the client side and is therefore limited in scope. They don't provide enough information about your entire application. Back-end performance testing is really useful when it comes to catching any performance bottlenecks when your application servers have been exposed to high levels of load. At the same time, front-end performance testing can catch issues related to browsers only, which can be skipped entirely from their protocol level. This is why a mixture of both is key.

Moving on, I want to talk a bit about performance testing tools because there's a variety of tools out there that are available to support you with your performance testing needs. From a front-end perspective, tools such as Lighthouse, Google PageSpeed, SiteSpeed.io, WebPagetest, and even your developer tools can help.

Other testing tools, such as Playwright and Cyprus can also offer ways to measure front-end performance. Then if we go to the backend tools, there's also JMeter, K6, Gatling, Torus, Locus, and also Artillery, just to name a few. These tools predominantly perform performance testing, most commonly load testing on a protocol level. So as you can see, you would need a combination of different tools to test your front-end and backend.

But what if there is a single tool that you can use for both? What if there is a tool that can simulate a browser-based test with a protocol-level test so you can understand how the front-end behaves during various performance events? This is where XK6 Browser comes in. XK6 Browser is an extension to K6, which brings browser automation and end-to-end web testing to K6 while supporting core K6 features. It adds browser-level scripting APIs to interact with real browsers and collect front-end metrics as part of your K6 tests. With XK6 Browser, this gives you the ability to measure how your front-end is behaving during certain events, which would be difficult to catch from the protocol level.

XK6 Browser, similarly with K6, is actually written in Go, but the tests are written in JavaScript. It's also great news for Playwright users because XK6 Browser aims to provide rough compatibility with the Playwright API. XK6 Browser is still in its very early stages, so it's been created as a K6 extension, which means that it's not included as part of K6 core yet. To get started with XK6 Browser, you need to install XK6 first via Go. Then you have to build a custom version of K6 with XK6 Browser binary added to it. Since K6 tests are written in JavaScript, there will be some familiarity already.

To demo a really simple test, I just want to visit a test URL. To create that, first I need to import Chromium from XK6 Browser. At the moment, XK6 Browser only supports Chromium-based browsers, but we also have plans to support Firefox and WebKit. Next, I have my export default function, which is our virtual user code. Anything that's inside the default function will be executed by a virtual user again and again, depending on your configuration. For now, this will only be executed once. I'm telling Chromium to launch a browser, and since I want to see the browser, I'm passing in headless as false. Then we're telling browser to create a new page. To visit our test URL, I'm using the page.goto method, passing in my test URL, and then wait until the network is idle. Finally, I'm just closing both my page and my browser. To run a test in XK6 browser, we just need to use XK6 browser followed by the command run, and then the file name. Let's see that in action by typing XK6 browser run, followed by the file that I want to run. In this example, I've saved it on a folder called examples. You can see that it has opened up my Chrome browser and it has visited the page. When that's finished, K6 prints out a summary of a bunch of performance testing metrics. Apart from the usual HTTP specific metrics that K6 already tracks, there are a few browser performance metrics that are now also added, such as browser download, first contentful paint, first meaningful paint, and so on.

For each of these metrics, you get an overview as to what the average time is, the max response time, or even the 99th percentile among others. This gives you an insight into how performant your website is from a browser perspective.

Let's make the script a bit more complex by automating a login functionality. Let's say that I want to automate typing in my login name and password, and then checking that I have logged in successfully. Let's launch an instance of Chromium, and notice that apart from the headless option, I'm also pausing in an option called slow-mo, which slows down input actions and navigation by the specified time. Next, I'm creating a new page. I'm visiting the same test URL again and waiting until the network is idle. After that, I'm using page.locator, which can also be interchanged to page.$ to interact with the given selectors and perform additional actions to type the username and password.

Now, for some context, a lot of the operations in Async are synchronous. However, playwright operations are Async. So we're also trying to support Async operations. Since clicking the Submit button will load up a different page, I also need to wait for that page and use page.waitForNavigation because the page won't be ready until the navigation completes. Once all the promises have been resolved, we can check that it has loaded a new page by asserting that the text content is equivalent to what we expect. Then finally, I'm closing the page and the browser. To see that in action, let's run our tests. Similar to the previous execution, performance metrics are also reported. But there is also a check here indicating that the assertion passed.

Now the real power of XK6 Browser shines when it's combined with the existing features of K6. XK6 Browser allows for mixing browser-level and protocol-level APIs. You can have a scenario where you want to simulate the bulk of your traffic with protocol-level scenarios. And at the same time, you can have one or two virtual users accessing your website on a browser level to collect front-end metrics such as DOM content loaded or first contentful paint.

To see that in code, first I need to import the relevant modules from K6 as well as XK6 Browser. Next, I'm configuring the behavior of my test using options, which is built in with K6 already. Here, I can run two scenarios concurrently. My first scenario is for my browser tests, while my second scenario is for my protocol tests. I'm using the constant vUSE executor for both scenarios, which will introduce a constant number of virtual users to execute as many iterations as possible for a specified amount of time. In this example, I've set one vUSE for my browser test while I have 20 vUSE for my protocol tests.

Next, I have my messages function, which is my browser tests, and I also have my news function, which refers to my protocol tests. It's all in one script, and this allows for greater collaboration amongst teams, because if you have back-end teams already using K6, front-end teams can collaborate more with them and be more effective when doing performance testing. Here is a sample output from the test run, and you can see both scenarios are executed concurrently with various performance metrics reported.

Now, since I started my talk talking about Overcooked, I would also like to end this with Overcooked. Now, if we want to have better kitchen coordination, handle high amounts of customer orders, make sure that we don't have customers waiting impatiently for their food, and also provide the best customer experience, you need to have the right blend of front-end and back-end performance testing. Just some final words before I finish. Since XK6 Browser is still a fairly new tool, we need help from the community, so you're most welcome to try it out and give us feedback. Do check out our GitHub project, have a look at our examples, and play around with the tool. That's it for my talk at TestJS Summit, I hope you learned something new today, and thank you so much for listening.

I want to go ahead and start though by discussing the answer to your poll question. So beforehand, you asked a question around what K6 is primarily, is primarily known for load testing, but can do a lot of other things. The correct answer is unit testing, so unit testing is the only item that you cannot do. Yeah. Yeah, that's right, and it's like really interesting here to see the results because even though a lot of people voted for the right answer, there's still a few people who maybe aren't as aware that K6 can also do browser testing, that K6 can also do chaos experiments, and even contract testing in the form of a schema validation. So the main type of testing that it can't really do because K6 is more of a black box type of tool, whereas it's not really suited for units testing because you'd really want to go deep into the different functions, so you would want it to be part of your actual code. You can do it, but we don't really recommend that you use K6 for that because obviously there's better toolings that are much available for that sort of type of tests. Yeah, it sounds really helpful though to be able to have such a versatile tool in K6 and being able to cover all those different use cases without having to stand up separate tools for each of those use cases. So yeah, really, really interesting stuff.

And so I also wanted to ask you back to our first poll question that we had for the audience about what is your favorite testing framework? And no pressure, but I'm curious what your answer is there. Yeah, I'm not gonna lie. It's my favorite framework is really Cypress. So before joining K6, I was a Cypress ambassador and it was the first framework that I've used so after I was on my maternity leave and it was just really great in a sense that it was easy for me to get started, easy for me to install, and the visual test runner was really helpful if I need to debug the test. So I think for me that developer experience is a really great thing that Cypress can offer. So, yeah, I'm a big Cypress fan.

Yeah, and so, yeah, you know me, I can't hide it either. Obviously a really big Cypress fan and talking about the different use cases and the developer experience, right? Being able to leverage these tools for so many different types of things. Cypress and K6 are similar in that space.

So we are having some questions come in from the audience. So we'll make sure that we answer those for you. First one is actually about how is browser level performance testing different from protocol level? Yeah, so I think if I go back to one of the slides that I have, so I've differentiated performance testing as either front end or back end performance testing. So when we're talking about browser level, it's really about testing the front end that we're trying to verify if, for example, our website is fast enough from the point of view of a single user. It also has distinct metrics, so it's all about like the browser performance metric. So an example metric could be how long did it take for the browser to render an entire page, or how long did it take for the page to be fully interactive? Whereas protocol level, so that's the most commonly used case, let's say for load testing API. So with protocol level, rather than using a real browser, we're using protocols such as HTTP to simulate a bunch of loads. So we're interested in testing our server response times and ensuring that when multiple requests are sent from different users simultaneously, that our backend servers and our databases should be able to handle that load. When it comes to load testing, obviously protocol level is a really popular choice because it's less resource intensive because if you think about browser level and load testing, even though with XK6 Browser, we now have that capability to spin up, you know, browser like virtual users. We still need to consider that with browser level load testing that it could be quite resource intensive because we don't really want to spin up like 1000 browsers just to simulate like a load test because that could crash like the servers that we're using. So, we have to have a balance of doing browser level and protocol level as well. Right, and that's something that we kind of see a lot in that in testing, it can not be as real world as it may be in production, right? And so it sounds like when you have kind of the separate metrics as well for browser and protocol, and you're testing them both, then you can also maybe be able to better identify where there are bugs or understand where issues may arise because you've already put it through the paces, so to speak, in a test environment. And so if something goes wrong in production, you can go back and you can evaluate that with that context already in place. Have you seen that be a benefit as well? Definitely. So, one of the use case that I've mentioned during the talk is that the real power that for example, that we're trying to communicate with xk6 browser is you can now simulate, for example, you can run a bulk of your load testing on the protocol level. So, let's say you want to create like a thousand protocol level requests. But at the same time, that wouldn't give you an insight to your user experience. So, let's say you want to check if any loading spinners are taking a long time. So, what you can do is while you have like a thousand load tests, like on the protocol level, you can have a couple of, you know, browser level virtual users just to check the overall user experience as well. So, you can now have, I guess, the full picture when it comes to performance. Because rather than having an isolated approach, you can then check, oh, what's happening in my front end if my backend is exposed to this, you know, like amount of like high number of requests.

Yeah. That sounds similar to actually kind of going into this next question from the audience. You may have touched on this a bit, but, you know, it states that there is an overhead involved in diagnosing CI failures, you know, for a single use case in, you know, end to end testing. With the K6 backend, you have access to the CI report and you can see failures in checks and thresholds. But knowing that you may or may not have a CI diagnosis tool like Replay or Cypress dashboard, when you have the K6 browser extensions CI diagnosis issues, how can you make the CI diagnosis aspect of that easy or easier?

That's a really great question. And I'm not 100% like closer to it yet because XK6 browser is still in a very early like beta stage. But one of the things that I know that the team wants to achieve in the future is that within K6 cloud, we have a feature called performance insights. So it can give you insights as to, which area really has the high number of like bottlenecks. So if you're utilizing, so if your CPU has experienced like a very high number of like utilization, then that performance insights will be able to tell you that maybe you can try adding some think time, adding some sleeps in your test. Cause again, we want to simulate what's happening in production as closely as possible. So we're still doing a lot of like beta tests in our K6 cloud. It's not like open yet, you know, to like the wider public, but that is something that, you know, we would have. So it can then give you some diagnosis as to like which areas have like a really like performance like bottlenecks issues. So we currently have that with the, with the existing K6 stuff already. So with the performance insights, it gives you information as to which, for example, servers have like degraded because of the high number of like load that you run. So we want to use, or we want to have the same sort of feature for XK6 browser for that. But for now it's not like fully out yet in the public.

Well, it sounds really interesting to be able to get those kinds of insights from a CI environment, you know, when is that something that we all struggle with is understand what's happening in those spaces. Another question here that we have from the audience is a question about the resulting output. So is there any way to configure alerts when certain metrics increase?

Yes, so you can use thresholds for that. So what you can do is, let's say, like one of your SLA is, you know, a specific like response time should be less than, let's say 500 milliseconds for like the 95th person cell. So in K6, we have a concept of a threshold where it's like a pass fail criteria. So if the threshold fails, then your test will be reported as a fail. And then in terms of like notifications, I think there is actually an extension that one of like our K6 contributors have written, I have to come back with the actual name, but because I've seen that there's like a list of extension in our XK6 ecosystem, that you can use it to send notifications to, I guess, like whichever, like sort of like platform you want, but yeah, the way to do it with K6 is you have to configure a threshold. And then after the test has finished running, then that specific threshold will then say if it's, you know, meet the criteria or not. Yeah.

Okay. And that probably helps with establishing your metrics too, because you assign those thresholds in advance and everyone's on the same page about what, you know, acceptable performance is. So that's always a good discussion to have. Another question here is, can you test Kafka producers and consumers with K6? I believe so, so we do have a K6 extension if you want to load tests on, you know, Kafka producers, consumers.

I would have to refer you to one of my colleagues because my main sort of expertise is around, you know, browser automation, but yes, we can load tests on Kafka as well. Okay, yeah, there's a followup question about whether you can fire custom events directly and test how the system reacts with Kafka producers and consumers, but maybe that might be a better question to hold off on. So if this person maybe sent me a message and then I can give you, or I can point you to the right person from the K6 team who can be much better suited to answer.

Yeah, absolutely. So we do have a very important question here, which is, which character do you main in Overcooked? Which character did I? Yes. Do I make or? Yeah, which character do you like to play in Overcooked? Oh, I don't know any of the names. I just let my daughter pick like any random ones but normally it's like the cute animals. She doesn't like one character who looks like this angry chef guy. So we tend to avoid using that character. But yeah, it's like a really interesting game because it's supposed to be a cooperative game, but you know, playing with a five-year-old, it's like very stressful, and which is why I wanted to relate it to my talk about load testing because once the orders have started to pile up, we can relate that to, you know, if you haven't tested your servers properly, you just get like a lot of queues, a lot of requests that you know, haven't been processed. And ultimately it results to a bad user experience.

Absolutely. So I've played Overcooked as well, and it requires a lot of communication, right, between, if you think about the front end and the backend, it requires a lot of communication between you and the other person to really be able to handle that. So I thought that was very fun, but awesome. So I don't have a main character either. I just kind of picked different ones. So I guess back to another question about K6, what was the main motivation for introducing the XK6 browser? Yeah, so I think just to give everyone some context, so we have this, you know, golden rule of web performance. I talked about it during, you know my talk as well. And basically it's saying that 80% of like bottleneck issues actually happens on the front end. K6 stayed away at the beginning, away from like real browsers, because they want to make sure that backend performance testing is like very stable. And I think now that K6 has definitely reached it's peak maturity, it's now very stable. We now have a lot of support from the community. We now want to shift our focus on front end performance testing, because we know that in order to have that hybrid approach to performance testing, we can't just do post-school level, we have to do both. So the main motivation really is to make sure that we provide our users a way of having a full picture of your application performance. And I guess you can do that already by plugging in different tools, by using different tools, but we want to see if we can try to have a single tool that can do that, that we can use the same script, and we can use or we can leverage existing features of K6 that our users are already using. And now, we just want to focus on highlighting more of the front-end performance side as well.

That's great, yeah, and thank you for providing that context. So, when it comes to other kind of tools, or testing tools like Playwright and Cypress, how does the XK6 browser either compete with or complement or work alongside those types of tools? Yeah, so this is a really interesting point for me, because even when I started K6, we had this thing called Week of Browser Testing. So one of the things that I've spoken about is we don't want to compete with Playwright, we don't want to compete with Cypress, because the messaging that we want to share is we want to provide a hybrid approach to performance testing. Yes, you can do performance testing with Playwright.

Yes, you can do performance testing specifically front end with Cypress. But what if you want to have the same tool for backend as well? Then you would need to use other tools. So the angle that we're trying to communicate here is like if you want to have a single tool for a hybrid approach to performance testing, then XK6 Browser with K6 can help with that. So we're not looking to compete, we're solving a different, I guess, problem that our users are facing. And I guess the more tools we can provide to our users that can solve the problem, then I guess the better it is.

Right, I guess the answer is always it depends, right? Yeah. Depending on the use case, the product that you're testing, everything like that.

So you can go ahead, attendees if you do have any additional questions, you continue asking those on Discord in Q and A production track. But we'll go ahead and wrap things up there Marie. Thank you so much for answering all these questions and for your really great and interesting talk. I know that I'm definitely interested in checking out more about the K6 Browser and performance testing for front end and back end.

So thank you so much Marie. Again, you can continue to ask questions of Marie in the Q and A production track channel on Discord. Thank you so much. Yeah. Let's do it. We're good. Yeah. We're good. That's awesome. Thank you so much for watching this episode of K6 Browser. We will see you next week. We'll see you soon. Take care. Bye. Bye-bye. Bye-bye. Bye. Bye-bye. Bye-bye. Bye. Bye. Bye-bye. Bye-bye. Bye-bye. Bye-bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye.