Demystifying Memory Leaks in JavaScript

Hello, everyone. My name is Ruden Britschwurter. I'm a NodeJS TC member. I work as a principal software architect at MyConvolt, and I'm happy to be here today at the Node conference.

And today I'm going to talk about demystifying memory leaks. So memory leaks are often considered something difficult and hard to solve. But is that actually so? And how can we ease the process?

To get to that, I would first like to answer the question what a memory leak actually is. And I'm consulting Wikipedia to answer that question. So memory leak is when a computer program incorrectly manages random access memory. In a way that memory, which is no longer needed, is not released. As such, we are going to stack up more and more memory over time. And the program, in its worst case, might just crash because there is no more memory to allocate. And that's really the worst-case scenario. Or maybe you're in a cloud environment and you're going to have to pay much more money because you have an auto-scaling active and more and more memory is allocated there.

So here we have a graph which clearly shows you how memory leak would look like in comparison to the memory usage and the time the program runs. And the yellow line is a perfect program pretty much. It starts up and you allocate some memory and then there are some ups and downs and this is perfect for each program pretty much. Sometimes spikes might be higher or going a little bit down again, but on average it's the line, the flat line. And the blue line is clearly a memory leak because over time you allocate just more and more memory without actually freeing it again. And this is bad.

So how do we handle memory in JavaScript in particular? Because we do not have to worry about it, right? This is all done transparently and this is perfect. And memory is just freed. Why should there be a memory leak in the first place? So memory is divided into a stack memory and a heap memory. The heap memory is a dynamically allocated memory and the stack is done by the operating system in a typical case. Each thread has some stack memory. It is the last in-first-out algorithm, also called lethal. It's very simple, it's super fast and the memory that is allocated on the stack is automatically reclaimed as soon as the function exits. When we compare that with the heap memory, there is a lot of things going on because the stack memory normally only contains pointers to the function that is currently running. And if you push onto the stack, the currently running pointer, then we get to the heap, which is the dynamically allocated memory. And here we have a nice overview from V8 that is done for the Times of India.

There is a lot of things going on. JSObjects are allocated, we have JavaScript source code, optimized code, we have regular expression code, strings, and much, much more. So, this is the main memory for our program in this case.

And the heap memory, in V8 in particular, is divided into three areas. Again, we have the young generation. So, as soon as we allocate a new variable, let's say we say let the pool is the string test, then you are going to allocate the memory test and it's going to be put into the young generation. This memory is relatively small. Mostly JavaScript will have intermediate variables that you use to compute the next value immediately. And you do not need the variable as soon as you compute your next value. This is all done synchronously. So, we want to dispose all of that memory that we do not need as soon as possible. So, this young generation will only survive one so-called scavenge run. This is the first run where our program to create a memory is going to try to get rid of those not-anymore-used variables. And if any variable survives that run, it is then pushed into the intermediate generation. And if it survives the second run, then it's pushed into the old generation. This is the bigger part of the application. It's normally used for long and used variables. So, things that you reuse in each application that might hold pointers to a lot of things. And we use a different algorithm to free the memory in this case. And the algorithm used there is called MarkSweep. We start off from our root object. The root object in a browser would be the window object, and in Node.js, it's global. While in modern JavaScript, it would be in both, just global disk. And we use a somewhat like, similar to a recursive algorithm where we start off with the root object and we just connect each dot. We try to connect each node that is somewhat connected to their root object. All other nodes, all other variables, or allocations are going to be freed. And this should be mostly ideal to free all the memory that is not used.

So, I already spoke about that we as a developer do not have to worry about the memory that we allocate because it's going to be automatically freed. What is done in the background is a so-called garbage collector is run and that garbage collector has a lot to do and at some point it might actually even go haywire and not, does not work as we anticipated it would be. So we have to look into that a little bit closer.

What are typical memory leaks or issues? This is in not a specific order and not all are actual memory leaks, for example, the first two. So, a wrong data type is still something that we have to look into because this is a very typical thing I frequently encountered. Often programs use memory types that are not efficient for the task that you want to solve. And then, when you handle like this big data, you're going to allocate a huge amount of memory, even though you would not need that, by just using a different data type.

So using the correct data type can also help you to prevent your program to crash in case you once encounter bigger data. And the second one is that we also try to run a lot of things in parallel. All of us probably love promises and async await, and we use promise all to improve the performance of the application, because normally, like with Node, things are all run on a single thread and we still want to do multiple things at a time, pretty much, to have multiple remote procedure calls. But when you just use too many of those at the same time, we also have to allocate memory call all of these steps. And when they are not yet done, they might also just cause an exception in your program because there's no more memory left. So this is also important to look into.

Now let's go to the first actual memory loop, event listeners. So something a lot of people know is in browsers, like some while ago, it was typical to just attach event listeners immediately to a specific DOM element in a list that you would care about an event to trigger from. And then we sometimes have to just add lots and lots of event listeners to all of these elements. Instead, you want to have a single event listener at a top node, that would then listen to all the events coming from all these nodes below. And then you don't have to attach so many listeners because event listeners also add up in memory. And sometimes we just add a new event listener for each event incoming. This would be a program at arrow, but it's frequently occurring.

And more difficult is even a closures. Because sometimes closures prevent the garbage collector from knowing when a variable is not used anymore. And when we are able to dispose that memory. So this is a tricky one. Then file descriptors can also cause issues. Because when you open up a file, there's a limited amount of files you can open at the same time. And when we just run our program, it normally works, even though we do not close in the file after opening it, that just can cause issues as well. And so we always have to make sure to close it on both success and non-success. So you should always have like a file link that no matter if you are successfully opening in that file or not. Because sometimes there might be something in between and you still open it. But there was an error somewhere else. And you still have to close that one. The most tricky part is that in JavaScript, and when we look in Chrome or any Chromium browser and Node.js is also run with V8, then we have to know some implementation details.

I'm going to go into some of those in a bit. Here's an example for just how bad it can be if we read too much data in a single chunk as well. So we have an HTTP server and we read a big file, the user requesting a file. This is going to take a while and it's not going to be efficient and it's going to allocate a lot of memory because we first have to read all that memory into the application and as soon as it's fully loaded then we are going to start sending out that data to the user, not before. And when there is not a single user but a lot of users requesting the same route at the same time we might also have a memory problem because and then just the total available memory is not going to be sufficient anymore and the application is going to crash. Instead, we should just care about streaming the data in small chunks. It's going to be much faster and we don't have to worry about the memory allocation anymore. We just get the first part in the file that we want to read and no matter if it's like multiple gigabytes in size, we are still able to send it out immediately in let's say a few kilobytes. And as soon as it's done, it's going to be done. The memory in the line is going to be very flat. We don't have to allocate more than the small chunks that we want to read.

And with the file descriptors, we also cared about handling the error. We open up the file, we want to write something into it, but we never close that file again. And if we do that too many times, an application would also have a problem. So we should always make sure to close that file descriptor. And that's done in the final week blog, as I spoke about before.

The event listeners could look something like that. Let's imagine you have a database that is event based and you want to write something into it. And we have an event listener on data. So there is HTTP server, and it wants to connect to a database and it wants to write something into it. So we write into it and then there is going to be a response at the data event. But in this case, we made a mistake because on each user request, we add a new event listener to that database instead of just a single one upfront. And then we have a lot of event listeners in the end that are all very tiny, but it's going to end up as a blue line that you see earlier. And this is the worst one, pretty much. It's a very frequently seeing that is reported, both on the Node.js issue reporting and on the V8 one because this is a V8 specific part that you just have to know about.

Strings are, like I mentioned, we have this very long string, a million characters long, and we just want to care about the very first 20 characters. So we slice out the 20 and normally you would imagine the rest is just read because we don't use it anymore. Instead, we just have reference because internally in V8 there's a lot of optimization going on and it tries to be very fast and memory efficient at the same time. It's a heuristic that is used that might not always work as anticipated. Let's imagine we have the string test and the string ABC and you want to compare those. Instead of allocating a new memory chunk that would consist of test A, B, C, it would just allocate a new, very tiny thing and that points to the test string and to the ABC string and it would say, I'm the combination of both of these. It's like a tree structure that you could imagine. When we want to slice something out of it again, it's also just going to point to the original string and says, hey, I have this starting point and this endpoint of that string, but it's not going to free the rest of the string even if it's not used anymore. And that could end up in a bad situation where we litter our program over and over and as we all know, we should not litter our environment. It's still difficult to prevent, but we should care about it. Otherwise the application might just crash and this is the worst situation. And if it happens, well, we have to deal with that situation and dealing with those memory leaks can be very troublesome. We might even get some anger issues and you spend a lot of time looking into something without really knowing what to look for. So we have to know how to identify the spots that cause that memory leak. And this is the most important part for me. So this is where we are going to focus upon now, how to detect and fix those memory leaks.

There are a lot of tools and also flags to identify memory leaks in Node.js. The first one is a dash dash inspect that you can just pass to the Node.js runtime. And then you're able to connect, for example, to the Chrome inspector, but also to other tools. And we also have a flag that is called trace GC. With the trace GC command, it is possible to see what the garbage collector does at a different time. It's also interesting to know about.

Sometimes it is very difficult to really identify the bug. And then you might want to use the dash dash abort on uncaught exception, because this is automatically going to cause a so called heap dump. And heap dump is the state of the memory at that current point of time. So when the application crashes, due to that exception, we can just then look at what the application was memory wise at that state. And there are more tools like LM node to look into those. If you care about more tooling, just check out this website, nodes.js.org guides debugging getting started, and you will find a lot of information to do that.

And I want to show you some coding actually. For example, for the string one. So in this case, I have the small program and I'm using the V8 module from Node.js and we have a variable that is count. We just have a set to zero and we have a variable pool. We have the function run, and this is going to do some things that we are going to look into in a second. We have this interval. It runs every millisecond. The run function is triggered every millisecond and then we create snapshots every two seconds. We know what the state of the application is at that point of time. And here we have this inner function and it is going to reference the original one, while original is again going to reference pool and pool is updated on each run. So here we just confuse the garbage collector due to the way these variables are all connected with each other. And let's see what would happen in this case. This would be this one. So I want to check that program out. So now I just open drone inspect and due to the dash dash inspect flag, I'm able to now see the remote target. So I'm going to inspect this. And here you see this overview, select profiling pipe, heap snapshot, allocation sampling, etc. And I want to look into some of these heap snapshots.

So I'm going to load one. And as you can see, the program creates new ones over time. So let's open up the first one. And let's open up at the last one. And I'm also going to stop the program. Now we have these two, because here you can look into it and there is like a lot of data going on. And it's not really sure what to look for. But we are able to compare it by just clicking on comparison. And now we can compare one of these heaps, not just snapshots with the other one. And then it's way easier to see where all your memory went into. For example, here we can see there's a big difference in the size delta of 2.4 megabytes. So this is clearly a point that you want to look into. And when we look into it, we see there's a lot of strings. These strings, again, when we look at them, it's first in this variable, in this string. And the string is in a specific object at that memory point. And this is in original and system context. Okay, context is in funk. Ah, this is something we know about in our program. So we know funk was here. This is our funk. And then you can just identify the actual cause. We now know that it has something to do with funk. Even though you don't see it immediately, we definitely know we allocated way too many of these strings because you can see them all in here. And they're very big. And they are not free. So you do know, you have to look into that.

My time is running short, so I have to hurry up a little bit. But I want to say this is also possible to use in a browser. Because this tooling is just available there as well. And for example, you can start it.

Click on it. And by clicking on Inspect, and then going to memory. And here is the overview of that as well. I want to show you a little bit more of another program right quick. For example, this is StringCrash. So here, I have that problem that I had spoken about with the V8 one. We can look into the code. This is just doing exactly what I've spoken about earlier. And the program crashes very fast because it cannot allocate more memory.

All right. So steps to fix the memory leak. First of all, please monitor your memory. Use, for example, an APM or something like that. And then you know how your memory profile looks like. Profile it with tools like the Inspect one. And take heaps of snapshots. But be aware, they are very expensive. You do not want to do that in production in most cases. Always try to do it in a controlled environment that is the airport testing. If you have to do it in a production setting, then you might do it with a very strict setting in a timeout, only a specific container or something like that, to only do it very infrequent. And then you can compare such heaps of snapshots. That's one of the most efficient ways to identify the leak out of my perspective. And then you can just identify it and fix the problem as soon as you know what it is all about.

There are lots of tools that may help you. You can check it out online on the website that I have spoken about. Memory is expensive, so not only cost-wise when you have to pay dollars or whatsoever currency for it, but also in the performance way, because when you allocate more and more memory your system is going to slow down. So you really want to prevent that. Keeps snapshots are expensive for a similar reason. It is going to be like a frozen application at that point of time. It can't do anything else while gathering up all the memory of the application.

Carefully choose your data type and your data structure, and only restart your application at the last resort. Try to fix the actual issue instead.

So thank you very much for being here. I want to end with one of my favorite quotes from Ken Thompson. One of my most productive days was throwing away 1,000 lines of code, and I can very well relate to that. Have a great day.

Hello. Hey, Ruben. Good to see you here on stage with me. Honored to have you, of course. So what do you think? 71%, is this the percent that you were expecting? Pretty much. It's really a common thing that people encounter. And it's something that often people just try to work around instead of really digging into the problem and to fix it. Like, I've seen applications to be restarted once a day, at least, because they had a memory leak that they were not able to find, or that they tried to just reduce memory usage in general or increase the memory that they were able to use, and things like that. And before they had to restart the application, that's something, obviously, that's not nice. You have to pay a lot of money for it. So fixing those is really important. Yeah, it really is. And yeah, well, rebooting is not really a durable solution, of course.

I want to go to the first question from one of our audience members. It's a question by I Am. The question is, how would you best handle the string slice example to avoid the original string being kept in memory? So in this case, you have to know about the internals of V8 pretty much, and to understand how these things work. And there are some operations that would flatten the strings. So they would indeed allocate a complete new memory chunk for the path that is required for your use case. And then there is no hard reference anymore to the original memory that you allocated, and then it would be freed. There is a library out there that is actually doing that as a very simple operation. Originally, it was just converting the string to a number in between. Then it would be internally cast pretty much. And it tries to create a new string internally in V8. It's just using a trick to make that string internally a different data type. And it's called flat string, written as F-L-A-T-S-T-R. And you can have a look at it on NPM. Yes, exactly. You can have a look at it there, and just check out some benchmarks. But mostly, you should just be careful about how you allocate strings to prevent it in the first place. Try to avoid it, of course. But instead of patching it later on.

Next question is from CreoZot. Is there any specific tools for identifying memory leaks you can advise? So I did point out the web page that you might go to. And there are a lot of different tools that you can use. It's on the Node.js website. And I can highly recommend you have a look at that.

All right. The next question is from Alexey. How to inspect and identify memory leaks in AWS, Lambda, or Google Firebase functions? It should be similar, anyway. As long as you have access to the, yeah, it should be. Well, first of all, you have to identify the leak, that you have a leak. That's normally what is the monitoring, too. You normally monitor your resources. That's the first part. You use, for example, an APM for that. And as soon as you identify that there is indeed a leak, you want to run the code in any environment that you have full control over and that you are able to introspect. And you are able to, for example, always create heat dumps in any environment. So that would be one way to do that. That would be one of the ways that you might just use. I personally prefer heat dumps, and you should obviously try to not do it in production. Always try to do it on a staging environment, if possible.

Next question is from André Calazans. Have you seen or done automated testing for memory leaks? And do you know if there's any way to differentiate live leaks from real memory leaks? So I have not seen actual testing for memory leaks, but if you monitor it, then you should normally have a notification as soon as anywhere in your application memory is going out of hand, because you would normally set a limit for the memory that you're expected to use. And then, as soon as it reaches that point, you should have a closer look. And then maybe go back to your Git history and see what's changed in there. It's a manual process from there on, but at least you have a timestamp. It went wrong after that deployment. So in this case, you could easily use git bisect. And git bisect is a very powerful feature. So write a test that would trigger the memory leak.

So write a test that would trigger the memory leak. So you know at least what somewhere causes it, even if you did not yet identify it. And then you want to identify the code. So you know, OK, when I run this code, and then at some point, for example, the application is going to crash after a couple of seconds or so. And then you know the leak is there. Then you bisect it, and that's a little rhythmic thing where you just step through from both sides. And if it does not crash, then you would know, OK, this code is not a leak. And so you can bisect it until you find the very commit that introduced the leak. Smart.

Next question is from Alexei again. Can large JSON objects be kept in memory? Sorry. Can large JSON objects kept in memory cause memory leaks? The JSON structure is not in particularly more causing memory leaks than anything else as a data structure. So yeah. But I think Alexei means that is it a problem if a JSON object becomes too big. In general, there is a hard limit in the data types, for each data type. Very much so. For example, for strings I believe, if I remember correct, it's roughly 2 to the power of 28 characters that a string might be of size. And as soon as you exceed that limit, there would be an error. And we would probably have that case if you have the JSON as a string. And if you have it as an object. I don't know of any limit in this case. There might be one. And there is definitely one for some data types. But luckily, I do not normally reach those limits.

Lucky you. Last question that we have time for. And it's a question from George Turley. Since we're talking about memory leaks, what was your worst memory leak you encountered? And if you were able to find it, how did you fix it if you were able to? So the worst one I personally encountered... it's already a while back. It's probably the one that I spoke about earlier was restarting the application once a day. I was using the application, I was developing it, and I did not have the possibilities to debug it. But it was definitely the one that bugged me most. I... I'm not certain anymore about my personal one that I fixed. I was lucky in the past recent years and did not personally encounter any of those. Let's keep it like that, then, Ruben.