The State of Node.js Core

Hi everyone, thank you for coming to my talk. Today I'm going to be talking about the state of node.js core. Right off the bat I wanted to start off with some download metrics. These are metrics for the past roughly two years and you can see from the trend that the number of downloads from nodejs.org continues to go up and to the right, with the notable exception of two big traffic dips related to, you know, like the Christmas holiday and things like that, but yeah, node is still trending up and to the right.

Next I wanted to talk about the ecosystem, so this is a graph from modulecounts.com which is comparing ecosystem size of npm compared to some of the other popular package registries out there for things like Rust and Java and Ruby and what not. This is not necessarily representative of node-core, but of the larger ecosystem, so npm is the only package manager on this graph that has over 2 million packages. JavaScript is by far the largest ecosystem in the world, and it does come with a few disclaimers, so npm does host things that are not just node.js modules, so there are other things on there. Not all modules are of great quality. There are some really nice packages out there. There are some other ones that probably don't get used a lot or shouldn't get used a lot, but just for a general trend of the ecosystem health, I think this is pretty representative.

Next, I wanted to talk about Node's long-term support schedule, also known as LTS. Back in 2015, Node created a LTS plan to balance individuals and developers with wanting to land as many features as quickly as possible with enterprise users, which value stability over the course of years, not just days or weeks. We came up with this plan where twice per calendar year in April and October, we would release a new Summer Major from the project. You can see here in the graph at the very top, the unstable bar labeled Main. That is where all active development in the project goes. Patches, new features, breaking changes, everything lands on the main branch. Then we maintain several release branches that we cherry-pick commits back to whenever we do a release. You can see here that right now we have Node 14, 16, 18, 19, and 20 as active or upcoming release branches. This year in April, we'll release Version 20, and then in October we'll release Version 21. So we do odd-numbered releases in October, even-numbered releases in April. The latest major release will become what we call the current release line for six months. So right now, Node 19 is the current release, and then after that six-month period, if it's an odd-numbered release, it'll go into a shortened maintenance period where it'll receive things like security fixes and whatnot, but it's really just kind of a grace period for people to migrate off, whereas the even-numbered releases go into what we call active LTS for a period of a month, or I'm sorry, for a year, and in active LTS, you still get everything but the breaking changes that land on the current release line, but we keep them in current for a few weeks to make sure that they're stable and there aren't any major regressions, and then we start backporting them. So you get the same changes but just at a slower pace and kind of with an increased guarantee around stability. After those 12 months, even-numbered releases go into maintenance mode for a period of 18 months, so this is to give ample time for large enterprises and whatnot to migrate to the next, whatever the next release line they're going to target is. So this is kind of an overview of all of the release lines right now.

Next I wanted to go and dive into each individual release line in a little more detail and see what's going on there. So Node 14 is currently in maintenance mode. It goes by the code name Firmium, so all of the LTS releases get assigned a periodic table element, so Firmium in this case. I think we started with Argon and Node 4, Boron and Node 6 and so on and so on. Some letters don't have periodic table elements, in that case we just kind of make some up.

That's just a fun fact. Back to Node 14, it will be end of life at the end of April, so you can probably expect 1, maybe 2 more releases between now and then, but if you're using Node 14 you really need to start migrating off now.

I would recommend aiming for Node 18 if possible, and the reason for that is Node 16 is also going to be end of life this year. So, Node 16 is known by the gallium code name. It should have been end of life in April of 2024, but it's only going to be supported until September of 2023. So, that is a 7 month difference, but there is a good reason for it. So, the version of OpenSSL that ships with Node 16 is also going to be end of life, so as a project, we had to balance, do we want to keep supporting Node 16 just because, or let it go away instead of asking people to run with what could be a possibly insecure version of OpenSSL by the time it hit end of life.

So, I have linked to the official blog post at the bottom of the slide here, if you want more details. But in general, I would just recommend, if you're migrating off of Node 14, don't go to 16. If you're currently on 16, start targeting Node 18.

So, next I wanted to talk about Node 18 briefly. Node 18 goes by the Hydrogen code name. It's currently an active LTS and it will go into maintenance in October of this year. And then once it's in maintenance, it'll be supported with security patches and things like that until April of 2025. Node 18 is very stable at this point and it actually has a lot of features that are not in Node 14 and 16. And personally, this is what I would recommend running in production. This is what I'm using in production at work. Like I said, very stable and definitely what you want to be targeting now.

Now there will be people always who want to use the latest and greatest things in production. So that would be Node 19 at this point. Node 19 does not have a periodic table element codename because it's never going to be LTS. It was originally released in October of 2022. It's going to continue being the current release line through April of 2023. At that point, it will have a few months of maintenance support and go end of life in June of 2023. As I mentioned, it will not enter active LTS. I do not recommend using this in production. In the past, we actually had a lot of instances where the current release line would have regressions that didn't make it back into LTS. That's not so much the case anymore. The odd numbered releases seem to be getting increasingly more stable, which is good. But I still wouldn't recommend it for production.

There is one big notable change in this release line. dTRACE, SystemTap, and ETW diagnostic probes were all removed. The reason for that is simply that Node is an open source project run mostly by volunteers. Nobody was volunteering or stepping up to maintain those bits of code. So, they were becoming a maintenance burden and they've been removed. There is an open issue inviting anybody who feels strongly about these things to come and support them again, but as for right now, they are not currently in the code base.

So similar to the graph that I showed at the very beginning, this is a breakdown of downloads by different Node release lines over the past two years. So, the blue bar which starts at the far left and then kind of drops off as time goes on is Node 14. That's kind of the expected behavior and what we want to see. So, downloads were increasing and then it moved into maintenance mode and they start to drop off. Similarly, Node 16, which is the orange bar, is still in the process of going up and to the right. I'm hoping that people, and it kind of looks like the trend is already starting in this graph, that people will start migrating off of 16 and onto Node 18. So, Node 15 and 17 are not shown on this graph because they had already reached end of life by the time that this chart was put together. But then, you can see the yellow line of Node 18 starting to pick up a little over halfway through the graph and it's slowly picking up steam. Like I said, if we redo this graph in a few months, I imagine that Node 18 will be substantially higher, hopefully more than Node 16. And then, the little piece of green at the very far right is Node 19. You can see it came into the world later than all of the other release lines. It hasn't picked up as much traction and hopefully we'll see it start to also die off.

So if you wanted to pick what version to run in production, I've already kind of hinted at this, but you should be definitely running an LTS version in production, especially if node is mission critical, if you're making money off of Node.js in production, then you want to be as stable as possible. You also need to ask yourself if you can update easily and frequently. Being on the bleeding edge implies that there could be blood. If you're insisting on running Node.19 in production, make sure that you're able to update easily because things move a lot faster there. And then you actually have to ask yourself if you really need all the newest features. Some people actually do, but oftentimes the answer is no. You don't strictly need the newest things and you can wait a few months. These are just recommendations. You can do whatever you want to do, but this is how I would recommend proceeding.

So next I wanted to talk about some of the new features that have made their way into Node via the V8 project. So the way things work is when there's a new JavaScript feature, TC39 creates a specification for it. Browser or JavaScript engine implementers like V8 will actually build out the new features and then once Node.js upgrades to a version of V8 that includes those features then you know we receive the features just by default.

One thing that we did have to kind of add some custom logic around in Node is enabling shadow realms. Shadow realms are kind of the language's official solution to things like Node's VM module so you can kind of run code you know off in its own realm, send inputs in, get inputs out. It's like VM, it's a little safer than Eval, but because it's still behind an experimental flag in V8 we had to add an experimental flag in Node to kind of unlock those features.

We also have a couple new methods on arrays. So find last and find last index are similar to find and find index they just start from the back of the array instead of the front. There's been a number of API additions and improvements. If you're using the internationalization APIs so locale has been improved and then the number format and supported values of method have also been added. There have also been some performance improvements related to class fields and methods. Typically when new features are added to V8 they're not fully optimized. So class fields and methods landed in V8 we had access to them in node but we still avoided using them because they were slower than alternatives using things like symbols and whatnot. But the performance there has improved and we can fully recommend actually using those in your code now.

And then a big thing that has recently come is the JavaScript promise integration. And this is specifically targeting people using WebAssembly. So the way things have generally worked is you call from your JavaScript code into WebAssembly and WebAssembly would do some synchronous operation. If it needed some custom functionality you could call back into JavaScript. But there was no real way to have asynchronous integration between JavaScript and WebAssembly. With so many of the new JavaScript APIs being promise-based or just asynchronous in general, there was kind of a disconnect there. So this is actually a huge quality of life improvement for people who are using WebAssembly and Node.

Next I wanted to just highlight a few of the APIs that we've gotten based on web platform compatibility. So Node does try to support, where it makes sense, a lot of the same APIs that you can get in a browser. So a big one that people wanted for years was Fetch. The project did finally make Fetch happen. Along with Fetch came a lot of other APIs like Request and Response and FormData. We have Web Streams, which are kind of the browser's response to Node.js streams. And right now they're not highly optimized in Node, but they're improving a little bit at a time. We have Web Crypto, Structured Clone, which is really nice for being able to copy objects. And I'm not gonna list all of the different things here, but we are constantly trying to improve our compatibility with the web platform.

So another big feature that landed in Node18 is a built-in test runner. So there are a few ways to use the test runner. So there's the node, colon, test module inside of Core that you can require.

There's also a CLI runner, which is enabled with dash, dash, test. And if you're using an older version of Node that, you know, this hasn't been ported to, like right now Node 14 does not have support for it.

Some kind of people were nice enough to take the code from Core, put it into a package on npm, and you can actually install it from there and get almost all the same functionality. So it supports a lot of the, you know, things that you would support a test, expect a test runner to support out of the box.

So it has subtests. It can skip over tests. You can specify only tests, so you know, you only want to run a subset of your test suite. Lifecycle hooks, such as before, before each, after, and after each, and things like that. And it does also support mocking out of the box.

So it doesn't currently support mocking of entire modules. Because there are some limitations around ES modules there. But you can mock functions, methods, and things like that. It does have built-in code coverage out of the box, which you can enable with the dash-dash experimental test coverage, CLI flag. And you can now also generate your own, or, I'm sorry, write your own custom reporters.

So there are a few that are built into Node. By default, it's going to be TAP, which stands for the test anything protocol, which is a format for expressing test results, which is used, among other ecosystems, not just Node.js. And that's going to be sent over standard out, but you can customize it. You can use SPEC, which is a more human-readable format. TAP, the little format where it just shows dots with Xs. I don't know what the name of that is. And then you can use the test reporter destination flag, where you can actually say where you want your results to be sent to. As I said by default, it's standard out, but you can use standard error and other things.

This is just a quick picture of what a test looks like. We support synchronous tests, asynchronous tests, callbacks, etc. On the left, here is what you would get if you ran the previous test and generated TAP output. On the right is what you get with the SPEC reporter. If you wanted colors and things that are a little more human-friendly, you could use that.

From the TestRunner also came another new feature called node-prefix-only core modules. The difference here, if you look at the top, we're importing the node colon test and node colon assert. Every core module can be imported using the node colon prefix, and that's nice because it lets tools and people reading the code know right off the bat when something is a core module.

At the bottom, you can also import the assert module and all the other core modules without the node prefix, but with the TestRunner, and probably with all future core modules, that's not the case. Node-test will get you the core module, just-test will try to load from user land, so node modules and things like that. It is an important distinction to be aware of, especially moving forward as there are potentially going to be more core modules that follow this pattern.

We also have a watch mode now, so it's very similar to nodemon. If you've used that dash dash watch command line flag, it will watch all of your imported files, and when any of those change, it'll restart node. It is integrated with the TestRunner, so you can run your tests, change some things, and then it will rerun your tests for you. And you can customize with the dash dash watch path flag which files you actually want to watch, although this is limited to macOS and Windows.

There's also util.parse.args. This is a simple API for parsing command line flags and giving them back in a data structure. This is actually going to be stabilized in node 20, so it's just something to be familiar with. One less thing you have to rely on NPM for. Very recently, we landed support for single executable applications. So this allows you to inject your application code into a Node.js binary, and then you can distribute that binary to systems that don't have Node or NPM installed on them. Currently, only a single common JS file is supported, but you can, you know, use a bundler or something like that to create that, that common JS file and then bundle your whole application that way. The Node project does have a helper module for this called postject, which is available on NPM. It was kindly donated by the people at Postman Labs. There are similar tools out there that exist, but the project doesn't officially test any of those. We also just very recently landed a permission system. So you would enable it with dash dash experimental permission. Once that is enabled, it locks down access to the file system, child processes and markers, and then there are various flags here below that you can use to unlock all or portions of those things. Wildcards are supported, so you can say allow FS read star, and that'll allow you to read anything from the file system. You can also use the programmatic API inside of your code. So process.permission.has, you can check if you have a specific permission, and then you can, you know, deny additional things at runtime. There is no way to then re-allow them. That makes sense because you wouldn't want to allow people to give themselves more permissions, but so you can always check, you know, what permissions you currently have, and it can be a little more granular. So you can see here we've disabled file system rights to a particular directory. We still do have overall file system right access, but not to that specific directory is what this example is conveying. There have been some other notable changes recently. Happy eyeball support, which will allow you to do DNS resolution with IPv4 and v6 at the same time. Streams helpers to go back and forth between node streams and web streams and things like that.

I'm not going to name all of them, again in the interest of time, one thing worth pointing out is we did have snapshot testing, the assert.snapshot, but people weren't really happy with the implementation, so we've removed that and hoping to revisit it to kind of refine it a little bit more.

Some potential upcoming features, just recently there was a pull request open to add foreign function interface or FFI to core. People are currently working on solutions for better TypeScript integration, quick building in proxy support to fetch, and hopefully promise-fing some more of the core modules, there are only a few left at this point. Again, these are all upcoming things that may or may not happen, so I think that they'll probably happen, but just I don't want to make any promises that I can't keep.

And then lastly I wanted to touch on some of the important links around the project, so if you have a bug report or a feature request go to nodejs slash node on github. A support question, you can either create a discussion or go to nodejs slash help. If you're curious about the release schedule, some of the things I talked about earlier in this talk, nodejs slash release has all of those details. If you have a security vulnerability that you found, please report it on HackerOne with the link here, don't report it on the public issue tracker.

And then finally, nodejs is active on the OpenJS Foundation slack, so there's a link here where you can join the slack. Nodejs and nodejs slash core are the big channels there, where the project is kind of discussed. And that's everything from me. Just real quick, wanted to take a shameless plug to the company I'm currently working for, Platformatic, which is helping to make API creation with nodejs a lot simpler for REST APIs, GraphQL, Metrix, all the things that are kind of undifferentiated, heavy lifting, we're trying to make simpler for everyone. And that is all I have.