V8 provides the ability to capture a snapshot out of an initialized heap and rehydrate a heap from the snapshot instead of initializing it from scratch. One of the most important use cases of this feature is to improve the startup performance of an application built on top of V8. In this talk we are going to take a look at the integration of the V8 startup snapshots in Node.js, how the snapshots have been used to speed up the startup of Node.js core, and how user-land startup snapshots can be used to speed up the startup of user applications.
Node.js startup snapshots
AI Generated Video Summary
The Talk discusses the Startup Snapshot initiative in Node, which aims to improve startup performance by adding new features and optimizing initialization costs. Startup snapshots, serialized binary blobs, are used to speed up startup and can be generated for both the core and user applications. Custom snapshots allow deserializing a heap from a specified snapshot, skipping parsing and compilation. The Talk also addresses misconceptions and limitations of startup snapshots, and highlights the different use cases for heap snapshots and startup snapshots.
1. Introduction to Startup Snapshot in Node
As mentioned, I'm Joy. I work at Egaleo and I work on Node and V8. So I've been working on the startup performance strategic initiative in Node for a while. The initiative has recently been renamed to Startup Snapshot as we have done the integration within Node core and we are enabling this feature for userland applications, which is what I'm going to talk about today.
So let's get started. So a bit of history. The Startup Snapshot integration started while Node started gradually dropping the old small core philosophy and adding a lot more built-in features. This includes new globals, in particular, new web APIs, new built-in modules, and new APIs in existing modules. These new features either require additional setup during the startup of Node core or require additional internal modules to be loaded during the startup.
2. Startup Performance and Initialization
So to keep the cost of the startup initialization under control, node core uses multiple strategies. First, we do not initialize all the globals and buildings as startup. For features that are still experimental or too new to be used widely or only serve a specific type of application, we only install accessories that will load them lazily when the user access them for the first time. And second, when building releases, we precompile all the internal modules to generate the code cache which contains bytecode and metadata, and we amp them to the executable so that when we do have to load additional modules, a user requests, we pass the code cache to V8, and V8 can skip the parsing and the compilation, and just use the serialized code when it updates, when it validates that cache. And finally, for essential features that we almost always have to load, for example, the web URL API, the FS module, which are used also by other internals, or like widely used timers, like time, widely used features like timers, we captured them in a V8 Startup snapshot, which helps simply skipping the execution of the initialization code and saving time during startup.
3. Startup Snapshot and Useline Snapshot Generation
With the default snapshot, the startup is generally twice as fast compared to launching without a snapshot. This gives us more sustainability as we grow Nucor while keeping the startup under control. Users can now create snapshots of their own applications, which is useful for applications where startup performance matters. The general workflow for building a snapshot is similar to building the core snapshot. Currently, the Useline snapshot only takes one file as input. There are two ways to generate the Useline snapshot: building null from source with the --null-snapshot-main configure option, or using the --build-snapshot-runtime option of the official node executable.
And we're still working on including more stuff there. Here. So with the default snapshot, the startup is generally like twice as fast compared to launching without a snapshot. For example, on like this MacBook, it goes from about 40 milliseconds to 20 milliseconds to start up Nucor itself. And from these, so on the left, that's the Nucor started up without the snapshot. And on the right, that's the Nucor startup started with the snapshot. And you can see like, even on just on the flame graph, there's less to be done, and it's obviously like much simpler, and it runs faster. So, yeah, and this also gives us more sustainability when we grow Nucor while keeping the startup under control. And we are still tweaking the internal snapshot to make sure that the built-in one contains just the right amount of essential features. But also, at the same time, the feature is now also available to users so that people can just create snapshots of their own applications. So this can be useful for applications where the startup performance matters, for example, like command line tools. In particular, if the application needs to run a lot of code during startup, or it needs to load a lot of system-independent data, these operations can be done when building the snapshot instead of being done at runtime. So the general workflow is similar to the workflow for building the core snapshot. So Null can take a user, provide a script, somewhere to do some essential initialization for the user applications, and it can run that script to completion. And after all the asynchronous operations are finished, for example, all the promises are resolved, Null can take a snapshot of the heap and write it somewhere, either into one binary along with the Null executable itself or as a separate block on disk. And when starting up again, Null can just get the pre-built snapshot from somewhere and then deserialize a user heap from it to skip the setups. So currently, the Useline snapshot only takes one file as input. So you'll have to bundle the setup code into one file. But we are also looking into Useline module support in the snapshot building script. So yeah, that's also coming. And currently, there are two ways to generate the Useline snapshot. So first is the tougher one, which is building null from source with the dash dash null snapshot main configure option, which tells the 2 chain to generate a snapshot using the provided user script and replace the default snapshot with a custom snapshot. And the final node executable would contain the user snapshot. So for example, we have a file here that contains something like global.js data. And I put some string there. Well, you can put many other complicated things, but this is like just an example. And then you go to no source directory, and then you build it with that configure option. And then the final executable that gets produced by the compilation process will contain a binary that already has the snapshot that contains this thing that you put on the global this. And another option that does not require building node from the source is using the dash dash build snapshot runtime option of the official node executable. So that might come handy if you just don't want to build node from source, which can take a lot of time.
4. Custom Snapshots and Runtime Synchronization
So by default, this generates a blob called snapshot.blob to the current working directory using the given script. But you can also specify the input output path with the dash dash snapshot blob option. And when you launch node using a custom snapshot, you can again use that dash dash snapshot blob option as a way to tell node to deserialize a heap from the specified custom snapshot instead of setting up a default Node Core heap. And that will help you skip the parsing compilation and execution of maybe your own code and help you run faster.
And now, there is another option that's work in progress, which is the new single executable application feature. And the snapshot can be layered on top of that. And this means that it will be possible to generate a snapshot and add it into a single executable with Node itself without having to compile Node from source. So, a quick preview of the current design is, so, the user can create a JSON configuration like that one, snapshot. You specify the main script with snapshot main, the path, and then specify where you want the output to be written. And then you use the official Node executable to take this JSON configuration and generate the blob. And then you copy the executable to your destination path because it's going to inject that blob. And you use, for example, the post-ject command line to the, is maintained official by Node to inject that blob into the binary, which is C there, single executable application. And then after you're injecting the blob into that binary, that will contain a snapshot and Node would just know that, oh, I have a snapshot embedded into this binary. When I am launched, I will just initialize that. And with this, you don't have to compile no phone source to use a embedded snapshot. So that's still a work in progress, but it's coming. It probably will land in 20. And we're also thinking about, instead of doing all this, we'll provide some kind of single line utility to just get a JSON configuration. And then it generates a single executable that you can just run without doing all this.
5. Design and Misconceptions of Startup Snapshots
That's also like there is a getter called it's building a snapshot that you can use from the API to determine whether the code is actually being run to build a snapshot. And another useful API is setDigitalizedMainFunction, so which allows us to specify a main function in the snapshot without having to pass another main script.
Different Tools for Different Means and Q&A
They used the same underlying infrastructure, like NVA to serialize a heap. But they are kind of different tools for different means. Heap Snapshots is designed for doing diagnostic on the heap, and Startup Snapshots is meant to be rehydrated, whereas Heap Snapshots is not, Startup Snapshots is. We've got loads of questions. All right, first question with the most upvotes. Can snapshots also be used to start up AWS Lambdas faster? Next question is, is there any chance of accidentally leaking sensitive environment variables to the snapshot during the build time? How much of a startup boost has been tested to be achieved for user land startup snapshots?
They used the same underlying infrastructure, like NVA to serialize a heap. But they are kind of different tools for different means. Heap Snapshots is designed for doing diagnostic on the heap, and Startup Snapshots is meant to be rehydrated, whereas Heap Snapshots is not, Startup Snapshots is. I think that's, there are so many snapshots in VA, so people can get confused a bit. Yeah, that's something that's commonly seen. Yeah, which also makes sense. They have similar names, people that are starting to learn as well.
We've got loads of questions. We went on my phone from having no questions to having absolutely loads of questions, so thank you to everyone in the room online who's been submitting them, wow.
All right, first question with the most upvotes. Can snapshots also be used to start up AWS Lambdas faster? So this is not something that, because in AWS, it's not something you, as a user, can control, like how Node gets started. So like the responsibility of using this kind of falls on whoever that runs Node. I think in this case, it kind of falls on Amazon. So, yeah, as a user of this, I was an embedder of Node, that's kind of out of your control, but if you are someone who can run Node yourself, or like if you can pass additional flags to Node, then yes, you can do that. Awesome, thank you.
Next question is, is there any chance of accidentally leaking sensitive environment variables to the snapshot during the build time? Which one? Oh, sorry, we can go through them together. It's the top one that is highlighted there. Is there any chance of accidentally leaking sensitive environment variables to the snapshot during build time? So in Node core, we have some internal assertions to make sure you don't leak them. So we are also considering exposing this to the use land. But if you're deserializing a startup snapshot, usually you don't get to see them. It's very difficult that you get to see them because Node will refresh all of them. So if you kind of like intentionally put something in there, that's possible. But also you can refresh them later. All right. Yeah, although I imagine if people were doing it, it probably wouldn't be an intentional act. Definitely something I've been guilty of. How much of a startup boost has been tested to be achieved for user land startup snapshots? So there is literally a TypeScript compiler in Node Core as a test fixture to test that we can snapshot the TypeScript compiler. So that's like you get still around, it's still in the range I mentioned about. It's like two times faster. Before it's like 200 milliseconds, for example, on my test machine.
Snapshot Performance and Limitations
And then when you turn the snapshot on, it's like 100 milliseconds. If you do a lot of initialization in your application, you can get like two times faster. The limitations of snapshots include async operations that need to be finished before taking a snapshot. Putting all the code in a snapshot to optimize the startup of the whole app is doable, but may not provide enough compilation cache.
And then when you turn the snapshot on, it's like 100 milliseconds. It kind of also depends on how much initialization you do in your application. If you do a lot, then you're going to save more because you're not even going to run code to initialize your application, you're just going to destroy the things. So yeah, in general, I would say you can get like two times faster.
Cool, thank you. We have time for one more and we have a bunch that have two ticks in. Let's actually go with the one that's right here at the top. What are the limitations of snapshots? For example, a TCP connection to a database won't be serialized or perhaps there's other things to consider as well.
Yeah, that's what I mentioned earlier. Like, because for example, that would be a async operation that needs to be finished before you take a snapshot because I'm pretty sure like it might be possible to somehow be able to deserialize a in-flight request, but also that's not currently a goal of this feature. So you kind of have to make sure there are no async requests before you take the snapshot and you need to resolve the promises. And yeah, those are the current limitation of these starter snapshots. Yeah, and hopefully start to build people's mental models around when it is appropriate to do this and when perhaps it may not make sense for a project.
Let's actually do just one more here. How good of an idea is it to put all the code in a snapshot to optimize the startup of the whole app? So one thing that you can do is just wrap everything in a function that you don't actually invoke when you build the snapshot. And then you put that function as the deserialize main function in the snapshot. So when you deserialize, it runs the whole function. So that's doable, but also you probably don't get enough compilation cache with that because if they're not on the top level, VA would selectively compile some of those but not all of them. There are some hints. So it's something that you can do, but also if you want optimized optimal result, you can try to put more of them at the top level. Yeah, makes sense, awesome.
Look, there are so many more questions. They fall down the bottom of the page, but we are out of time. So I will remind everyone, both in the room and online, that the speaker Q and A room is where to go to continue to ask questions about this topic. The physical space is out by reception to the left of the door as you're kind of facing to exit it and those online, you can use the Q and A room in the spatial chat, but please join me in a massive round of applause for Joy. Thank you so much, what a fantastic talk. Thank you. Thank you.