Making My Node.js API Super Fast

Hi everyone, happy to be here and welcome to my session about how to improve performance in your Node.js API and make it really fast. So before starting to talk about technical stuff, let me introduce myself. My name is Tamar. You can follow me on those links and contact me on those links. I'm really passionate about JavaScript and Node.js and my interest in those technologies started when I founded a startup of my own and I wrote the entire backend in Node.js and from there the rest is history.

And enough talking about myself, let's start talking about performance, which is one of my favorite things that I like to talk about. And today we are going to talk about how we're going to improve performance in your API layer. And we're going to revise, we're going to look at those areas, we're going to look at database. We're going to look at serialization and deserialization of your Jasons. We're going to talk about logging. It looks trivial to you, but that can impact your performance. The web framework that you choose and caching.

So let's start with database work. First of all, before we go into JavaScript related stuff, there are some things that you need to keep in mind for every technology that you work with, for every programming language. First of all, of course your DB queries have to be optimized. All of the indexes have to be very efficient and enable really fast fetching and efficient operations. And then you need to denormalize your data. What does it mean? If you have a table of 50 columns and you only need three, don't fetch the entire 50. Fetch only what you need, for example. And save the data in a way which is efficient. Don't save inefficient things. And then you need to think about solutions in the DB layer. That can be read-write replicas that can improve your performance. Working with a replica set, matching the DBs that support that, and sharding.

But we are in a Node.js conference, and after talking a little bit about generic tips, let's go into Node.js stuff. How can I improve my Node.js application-related database work? When we are starting to work with Node.js, we are starting to work with the first tutorial that we are all doing, Node.js, express and Mongoose. Let's start with that, and let's take a benchmark and see how that is going to perform. When we are going to do a benchmark, we have to keep in mind the following things. First of all, the measurements that are the most important is the latency. You need to look at the 99th percentile, not the average. Because when you are telling you have a contract with a third party, and you are telling them all of my requests are faster than this, you need to give them the 99th percentile in the latency. Not the median, not the average. That is really important. Second, measuring throughput. For measuring throughput, we have two important things that we need to look at. First is the average request per second, and then the total request. Those measurements will give you a good idea about how your throughput is improving.

Now we are going to compare Mongoose and Native Mongo Driver. I wrote a server that works with Mongoose, and that server exposed one post request, that is posting a person object, something really simple. The code is not something really interesting. We are building a server with Express and Mongoose, building a Mongoose schema of several parameters, and then that is the function that we are going to measure. Post, and then we are creating a new person object, and saving it as you can see here. The Native Mongo Driver code is going to do exactly the same. A post request, which also is posting a person object, and that is because we want to compare apples to apples. When we are doing performance benchmark, we want to compare things that are doing the same. Here is the code that is running with the Native Mongo Driver. As you can see, this is also doing insert1 into the database.

Cool. Those are the results that we have with Mongoose. You can see if we are going to look at average requests per second, we are around 5700 requests, and the total request is around 64,000, and the 99th percentile is 4 millisecond, but as we are using Mongo Driver, we can see that things improved hugely. We can see that the average requests per second went up to 9500, which is a lot. It's more than 30%, I think. The 99th percentile is much smaller, by 50%, and we were able to serve, in the time of the benchmark, around 30,000% more requests. As you can see, if we are going to see it in a graph, Native Mongo Driver performs much better than Mongoose.

We are going to compare total requests and average requests per second, and you are going to see that it is much better. The question is asked, why is it much better? Why is it performing much, much better? To make a long story short, Mongoose is a tool from a type called ORMs, and usually when you are using those tools, they are much more out there, they give you an abstraction layer above your database that helps you, but it adds an overhead of serialization, dereferentialization, sometimes inefficient queries, so usually most ORMs harm your performance. Those are good for applications in low-scale, medium-scale, but when you are going to high-scale, usually you are starting to work natively with your database, whether it is Mongo, or SQL, or whatever. Then let's talk a little bit about connection pools. What are connection pools?

When we are working with a database, then we want to have a pool of connections to the database that will be available, each one of them can execute a query, and when it ends, it goes back to the pool waiting for another client, another utility to use it. If we are going to compare results, when we are working with a connection pool to your database, and working without a connection pool, so now let's do that comparison, again, we are comparing apples to apples, and we are comparing the same server, and that server is doing that POST request that we spoke about. So, when we have no connection pool, let's look at the throughput. We see that the average request per second is around 8100, and the total request is about 81,000, wow, we have a multiply of 10 here, and then you can see that if we are optimizing our connection pool, making it bigger, making it more efficient to our machine needs, to our environment, then I succeeded to get around 15 to 20 percent improvement. You can see that the average request went up to 9500, the total amount of request is a little bit close to 100,000 requests, and if we are going to look at that with graphs, then we can see that when we were optimizing our connection pool, then the performance went up by 20 percent. So, we are looking at the total request, the average request, also the latency became shorter. So another thing which is important to keep in mind is to work with a connection pool, and make sure that you are optimizing it.

All right, now let's talk about efficient JSON parsing and stringify. So, when we are working with an HTTP, when we are using a POST request, when we are implementing a POST request, we are deserializing the HTTP request, the body of the request, and when we are implementing a GET request, we are serializing the response body and giving it back to the server. But serialization and deserialization is a synchronous operation that always blocks your event loop. I really hope that you have Node.js architecture in mind, that you are familiar with the event loop that is processing activities, processing work, and when we have a synchronous operation which is long, it's blocking it. And serialization and deserialization are exactly those things.

So, now I'm going to take the same server and replace the response serialization with another library called FastJSONStringify and let's see what we are going to get here. So if we are going to work, I'm implementing a GET API here, and I'm working with Express and I'm using Express.json, as you can see, the default serializer, and that is a function that I'm testing. And Express.json is executed when we are doing result.json. OK, so we are doing find one and returning the response. Now I added a little bit of addition of my own. I created another server. I used FastJSONStringify. That library needs to receive a schema, so I gave it a schema, as you can see here, and I created a middleware of myself. I replaced the default Express serializer with my immediate middleware, and that middleware is using FastJSONStringify, that custom middleware. And if we are going to compare those results, we are going to see that that gave me also around 15% improvement to my Express server. For example, the average request went up to above 10,000 requests per second, and also the total requests, those went up in around 50%. I was able to serve 50% requests more, so the throughput went up, and also the latency is smaller. So FastJSONStringify gave me the performance advantage here. But you know, that's not the only library that exists around. There are more libraries that exist, and I really recommend you to look for other serializers, understand what is more efficient in doing your test, and maybe yes, consider making this layer of serializing and deserializing your request and response content more efficient.

Alright, let's talk a little bit about logging. In a perfect world, our system is working perfectly, but actually usually in reality, there are a lot of problems, applications crash, there are random bugs, data is missing, so logging always helps us to debug, and logging is something that we all need in our applications. But logging can cause performance degradation.

So I was, or I'm sorry, it cannot, it will always cause performance degradation. So I was comparing one server with a POST request that is, you know, with no logs, and another server which have only one log line, and that's it. And look what we have here. If with no log, the average request per second is around 9600, then when we have only one log line, the throughput is down by around 15%. Look at here. The average request per second is around 8400. In the last results, we were able to serve like around 100,000 requests in the total benchmark. Now we're able to serve like 93,000. That is like 10% degradation.

So when you're thinking about logging, that's also something that you need to take in mind to take a library which is performance efficient. Different log libraries that are, you know, popular right now are, let's say, Winstone, PinoJS, and other libraries that I took for my benchmark comparisons are Banyan and Log4j. So when we're looking at the overall, we're gonna see that PinoJS is the library that currently gives you the less overhead over your code, meaning that that is like for now, for my comparisons, the most efficient log library performance-wise, and it would give you the best results for not harming your application performance. So we can see that in the average request per second, and we can also see it in the total request as you see that Pino, the server with Pino was able to serve the amount of requests which is closer to a server with zero logs. So if we're conclusioning that, so if you're thinking about performance and you're thinking about performance efficiency, so prefer PinoJS over other libraries. That can change, but that is like for now, the most efficient log library out there. Another tip that I would like to give you, which is important, is, you know, when we're doing logs, we have a lot of string concatenations, but the log library should manage this string concatenation. Don't do the string concatenation in your application by yourself. As you can see in the example which is below, because it seems like minor to you. What is actually happening behind the scene, that if my log level for now is like error, then that string concatenation won't be done by the log library. If I'm doing it by myself, it will always happen. So it looks minor, but really in large application, complex application, those things add up. So always prefer that the string building would happen with an interface, which is suitable for that from the log library, which receives parameters. Okay, now actually I was doing like tons of examples with Express. So the question is whether we can do something else, maybe not Express. I mean, Express is, to be honest, is kind of old. I don't know if you know that, but Express release version is version 4, and that was released in 2014. That was quite a long time ago.

I know that the first tutorial is Express, but we have a lot of other interesting players in the market. We have Nest, we have Festify, we have Koa. So maybe if we're going to take something else, that can give us an advantage that we need.

So I was benchmarking Express and Festify and NestJS, and of course when benchmarking, it's very important to, as we said, compare apples to apples. If I'm comparing a get request that saves an object, a person object with three parameters, I have to do the same thing with Festify and with Nest, the code has to be the same. And on the database, if we're doing insert one, we need to do insert one on the other framework. Okay, let's see what we get. So when we're working with Express, we were able to serve around 95, 94 hundred requests per second, and the throughput is around 100,000 requests per second. Latency, two milliseconds. Nest gave us something which is slower. Average request per second, 5,400. We were able to serve around 60,000 requests, but Festify currently gave us the best results. We were able to serve, as you can see, close to 200,000 requests per second. The latency went down to one millisecond. So Festify did give us the performance edge and here is the comparison. If we're going to look at it visually, then we're going to see that Nest was performing about 50% slower than Express, but Festify is currently around twice as fast than Express, at least from my benchmarking. We can see that with multiple measurements that was done. So actually, in most companies, it is a little bit hard, let's say it, to refactor your microservices code. Usually you get a code that you need to work on, and in that case, well, you have what to do. You can make your DB work more efficient. You can use caching. We're going to talk about caching right away. You can improve serialization, but if you have a possibility to choose a more efficient web framework, or if you have a possibility to start a new microservice with a different web framework in your organization, you should really consider that, because that can give you a real boost and edge. And, as we said, Festify currently is the framework which is performing the best among all the frameworks that I looked at.

Now, I hope that your mind is not blown, but we're going into the last thing that we're going to talk about, which is caching. So for that, I had implemented a server which is authenticated with JWT, and token validation is done in JWT authentication on both sides. The client has to validate the token, and the server has to validate the token as well. But I'll give you a spoiler. Adding a JWT authentication to the server has caused the performance to decrease in a really bad way. Let's see that. So that is the middleware that I added, and for every request I'm doing, JWT verify for the token. Okay, the code is not really complicated here.

So that is the middleware that is used. And look what happened. So from a situation where we're serving around 100,000 requests in total, we went to around 15,000. That is like 80% degradation. That is huge. So yeah, like things and also the latency, you know, went up to around 10 milliseconds. It was like massive. So to solve the problem, well, the best thing to do here to solve the problem is to use caching.

So those are the results when we have no authentication. That's what we got. We know that around 9,500, 9,600. Those are the results that we received when we added the authentication. You see that the average request per second went down to 2,500. And when we're using caching, you can see that we're really close to what we had in the beginning. We had like without the authentication, we had around 95 requests, 9,500 requests per second. Here we have 9,100. That is kind of close. And also we can see that the latency is back to 2 milliseconds, the 99th percentile. That's what we're looking at all the time. And here you can see it like visually. When we were adding caching, it is close to what we have when we're doing no authentication at all. So adding caching to the authentication layer solved the problem.

We can see it with different measurements. And we can see that the latency is back to what it was. That was the bottleneck and the performance problem was solved.

Types of caching that we can do. We can do in-memory but actually that is not really recommended. Because most of the time if you're doing in-memory caching, you cannot run your service in multiple replicas. You cannot create multiple instances of your service. So it's not recommended. Better to take technologies like Redis. Redis has other competitors like EDCD for example. But better take technologies like this. And in that case the cache is distributed. All your services are accessing the cache. And it's much better.

What you should cache. Or what is recommended to cache. DB queries, crypto stuff. For example we saw JWT. HTTP requests from third parties. Those kind of things. That would improve your performance a lot. Okay. We are at the end of this. Which is good. So let's do a quick summary of everything that we've seen. Optimize DB queries. Make sure you have efficient DB architecture as we've talked about. Better to work with native DB drivers in large scales. Have your ability to optimize your connection pool. Cache stuff when you need.

And also make sure that you choose an efficient web framework for performance. You should choose efficient log libraries. And you choose efficient serializer. That was me. I really hope you enjoyed it. And you can follow me on those links. And thank you very much.

Which areas should we improve performance in our services? And while Columbia is a great answer. A lot of folks are. Definitely. But people have said things like latency, request time, throughput, query optimization. So what do you think? Do these results surprise you? What would you answer to such a question? What would I answer? Okay. So first of all, it could be that the problem is maybe not in your application. Maybe it's like an environmental problem. Maybe if the machine is running in the cloud, then there are problems with another container. Which is cutting the machine, which is cutting all the resources. So you have to make sure that you're not into that situation. By the way, I think that optimizing your code is something that you do kind of less. You have to look, first of all, into your database. And see whether you need to optimize your queries. And you need to see whether your connection pool is optimized as well. And you need to also see your logging level. By the way, never run production applications in logging level in the bad mode. Always run them in error mode or warning, because logging can affect a lot on your application. So before starting to optimize your JavaScript code, there are a lot of things that you can do.

Fantastic. That's a great answer. But I want to also move on to the questions from the people who tuned in and have questions for you. So let's head over to those questions. And I actually think that's a good segue into what the first person asked.

Which is, what's the bigger performance bottleneck? The technology? Or the humans in the process who are implementing the technology? What do you think is the bigger issue for performance? Well, that technology can impact. For example, I think that Node.js is performing much better than other languages such as Java. Because of its unique non-blocking I.O. and event loop model. But I think that as humans, we also have to make sure that we are dividing our software into good architecture. We have to try to divide our software and microservices in a way that it can scale. So, it might be that humans will be able to... Humans are the bigger problems, to my opinion. Humans are the bigger problem. That's it. Like it's come to... Layer seven. Yeah, like database work which is not correct. Logging work which is not correct. Microservices architecture that is not scaling. Caching that is not used. That's why we need to welcome our machine overlords. Now is the time.

Awesome, that's a great answer. So what do you recommend to isolate performance bottlenecks in Node.js applications? What's your best tip? What's my best tip? By the way, I do recommend that if you start a new project, then try to look at new frameworks. Don't go to like, I don't know, express what everybody is doing. Try to take new frameworks that can give you that performance overhead. And try to work with asynchronous operations, by the way. Whenever you're talking about Node.js, that is crucial. Try to work with asynchronous operations. Everywhere that you have an asynchronous API, work with it. Work with Promise sol wherever you can. Always try to run things in parallel. Because, you know, when we're working with async await, we like to run things one by one. But if you can, run them in parallel to use like your entire worker threads, like worker threads power.

So do that. Run things in parallel. And if you need to do like CPU intensive synchronous operations, then use the core model of worker threads to run them in a thread pool. So that is like the Node.js specific performance optimization that are important.

So that, okay. So you mentioned that Express is the default. And do you have, that's also the question here, NestJS, Express. Like what about Fastify, other tools, what would you recommend? What's another non-typical?

I think currently Fastify, it gives you like the whole package. It runs much faster than others. It uses a much efficient serializer. In my talk, the reason that I showed the serializer is for people that work with Express and cannot change technology. So that could be like a quick win for them. But Fastify gives you like the efficient serializer, the efficient logger. It gives you like the whole package. So it works efficiently with a database, not with Mongo. So I think it is recommended now.

Amazing. Thank you so much for that awesome Q&A and your excellent talk and for joining our community. It was wonderful having you. Thanks so much, Tamar. Thank you very much for having me.