What's New in npm?

Welcome, everybody. Thank you for joining me today. My name is Darcy Clark, and I'm going to be talking about what's new in the NPM CLI. I want to give a big shout out and that's been running the conference so far and all the folks behind the scenes at DevOpsJS 2021.

A quick overview on who I am. I'm the engineer manager for the NPM CLI team at GitHub. I'm based here in Toronto, Ontario, Canada, and there's a bunch of other places that you can find me on the interwebs.

So, I want to start with talking a little bit about what NPM does or what people think we do. And I think the biggest thing we're known for is installing a whole bunch of packages, so this is our average package download count. I think last time I checked, we're almost at 125 billion package downloads monthly as a community, which is a pretty amazing mark. I think we had about 100 billion average monthly downloads back in September, October, October, around the 11th anniversary of the 11th year of NPM, which is pretty cool. We do more than just install packages and publish packages. So the NPM CLI supports also discovery, building and bundling your packages, testing, and a whole bunch of other features with a number of commands. And if you're looking for information on the 61 plus commands and growing, you can go check out our docs at docs.npmjs.com. You'll notice that they just recently got a nice new design, a refresh, and there's a ton of information about the config as well as the commands that we support.

So what has changed in the NPM CLI? Because you might be wondering, I switched tooling a while back. What has changed or what's new in the NPM CLI? Well, what's changed quite a bit is our team. Over the last couple of years, we've added and had new folks join our team. Currently we have a core team of myself, Roy Adorno, Nathan Lafreniere, and Michael Garvan, our GAR. We're also expanding this team and adding three new folks come next month. And we also get a ton of contributors, almost 700 contributors to the NPM CLI, which is pretty amazing. What's also changed is that we just recently shipped NPMv7 and went latest or sort of generally available in January 2021. So, in v7 we introduced a lot of new capabilities, including some improvements to legacy commands and also added things like workspace support along with much, much more. So, let's dive into some of these changes. So as a v7, one of the biggest things that you'll note is that we've started to install peer dependencies by default. Peer dependencies are not new by any stretch of the imagination, they've been around for about eight years. But what we noticed is that a lot of projects were starting to have to, and a lot of developers are starting to have to manually manage these dependencies, which is a problem. NPM is a package manager and we want to manage your packages. So in this example, you can see that I have two workspace projects that actually depend on different versions of react. These are essentially conflicting versions, and we don't quite know what we...

Previously, we would just leave these be and let the developers essentially manage these. As a v7, we will now try to do our best to install and resolve your peer dependencies. And if we can't, we'll print a sort of a warning with some actionable steps for you to take to essentially resolve these dependencies. And if you're looking for the actual logic that is running and doing this resolution, we've moved everything that we... Sort of the brains of the operations into a net new package that we call Arborist.

So moving on, we actually introduced, to sort of help with this change, we also introduced npm explain, or the alias npmy. And this command actually functions a lot similar to yarn's npmy and npm explain, where it will actually show you the reason why a package was installed. So in this case, I've run npm explain chalk, and asking essentially, why is chalk installed, and npm will let you know that in this case, I have a workspace project called b, and it actually includes chalk as a peer dependency, and that's the reason why that got installed. So some nice developer, you know, experience and developer tooling to sort of help with ergonomics around managing your dependencies.

Also, what's kind of new npm exec was introduced in v7. It's essentially the guts of npx. So if you've ever used npx, then this will be, you know, not very new to you. We added some safeguards with npm exec and npx. In v7, we now actually ask before installing a package we've never seen before, we actually ask you to confirm and have a confirmation prompt to ensure that you don't accidentally install something and run something that you didn't mean to.

So also as of v7, we've made significant changes to npm audit. npm audit got sort of a an entire refactor in terms of the logic, performance, and also ui. So now we hope that it's a lot easier for developers to quickly grasp what exactly is going on and what the flags are for the different dependencies that might have a vulnerability issued against them. And we hope that this is actually a way faster experience for developers and we think it's an improved experience overall for sure. And a net new command that we actually introduced just recently is npm diff. With npm diff, it works very similar to get diff in that you can actually specify a package name, a version or a file and you can essentially see the difference between those two things. This is great. In this example I'm essentially seeing what's changed between two different versions of my package called sleepover and I can see that the version was updated and the package json was changed slightly.

Another big change in npm v7 was the introduction of a new package lock schema. Previously we were on v1 of the package lock and as you transition or as you upgrade to npm v7 we will seamlessly upgrade your version of your package lock file to meet essentially this new schema. So there shouldn't be any problems with upgrading and if you have any problems with developers using v6 or v7 you can always say no save or provide the flag no save to not update your package lock files. So another huge win that we sort of accomplished with npm v7 was some improvements to the actual performance of a lot of the commands. Specifically on install we initially saw a sort of a degradation but we noted that we were actually installing more dependencies because of the changes in behavior that we had introduced with peer dependencies. So peer dependency resolution bumped out some of our install times because we're actually installing and resolving more dependencies so this was somewhat expected, and as you can see we've started to benchmark across different sort of scenarios and fixtures including installing peer depths. So that's that last benchmark there at the bottom. As we sort of move forward we actually saw that we made actual significant improvements against ourselves and are hoping to continue this work inside of the benchmark suites that we're running and are hoping to continue to focus on performance as a place for us to optimize and get better.

And so if you're ever interested in the work we're doing there you can go to npm slash benchmarks and you can go see the different runs that we're doing against the different versions of npm that we're building.

So npm v7 introduced support for workspaces. This was a huge win for our team. It's something that we wanted to offer to the developer community for a while now. And the basic support here for workspaces is just the definition of the projects within your root project.

So in this case, in this example, I've just defined essentially an A and a B workspace. And as of v7, 7.0.0, the only real command that understood or supported these workspace definitions was npm install. And I actually have a reference there to our docs. And there's references also back to the RFC that we originally created for this basic support.

So since then, we sort of broke out the work into another two phases. And the second of which is what we've been working on just recently, which is making more commands essentially workspace aware. And this second stage of this work, really encompasses all the other 60 plus commands that we have to define them and categorize them in terms of their workspace awareness.

So what you'll see is two new configurations that are actually going to be released in the next week or so, along with some other work that is associated with workspace awareness. So the first config that we have is workspaces. And it's essentially a bullion to be passed to any of the NPM commands to let you know that you'd like to run that command against the workspaces that you've defined. And then the second config is sort of a filtered command for the specific workspace you'd like to run the command against. So that's just workspaces and workspace. So ws and w are the aliases.

And if we see what this looks like in practice, so as of NPM 7.7, you will be able to run on any of the lifecycle scripts or lifecycle commands like run scripts, run test, start, stop, restart with the ws or w flags. In this case, I've run test with ws, and it goes and runs all the test scripts that are in the workspaces that I've defined in my project. And here, in this case, in this example, I've actually filtered just to run the test for the workspace A that I've defined in this project.

And workspace is pretty nice because it's very composable. You can actually define workspace many times. So, in this case, in this case, I've actually defined B and then A. And because of the way that this is implemented, they run serially. And you can see that B's tests run first and then A's. And if I were to define more and more workspaces, you'd essentially see more and more runs done.

So, this new feature, this new capability with working with workspaces is available as of NPM v7.7, which we're releasing in the next few days. 7.8 will have exec and init support and the rest of the subcommands becoming workspace-aware will come subsequently after that. If you're ever looking to get the latest version of NPM, you can install us with npmi npm-g.

We ship weekly now. You'll be seeing new patches, new miners, new improvements coming each week. I want to take some time not just to talk about what we have been shipping and what we are going to be shipping, but what does our pipeline look like, what's coming down the road.

In the quarters to come, the Q2 and Q3 of 2020, we are looking at registry protocol and package overrides as two RFCs that are in our back pocket that we want to essentially queue up the work for, including published prompts in Q3, which is another RFC that has been approved, and we just need to do the work. As noted earlier, we are doing workspace improvements right now, and hoping to finish off the quarter with also filtered or scoped installs, which is a really cool feature.

If you'd like to learn more about the registry protocol or the override specs and RFCs that are being proposed right now, you can go jump over into the npm-RFCs forum and go check out what those look like. And if you want to get involved even beyond that, again we run a weekly npm-OpenRFC call, which happens at Wednesdays at 2 p.m. Eastern, and that's a live call where we talk about the future, current and future of the project and discuss essentially future requests and things that we would like to get done. We also have resources for the docs that I referenced earlier, our bug tracker, events that we're at, that we're going to be at, or speaking at, our roadmap and other feedback resources at npm.community.

And that's it for me today. So just wanted to give you a big thanks for everybody that's been involved in the event and feel free to reach out at one of these links here below. Cheers.

Hey, good to see you. So we're going to look at the results of your question. Have you used the npm-cli previously? And 100% percent saying yes, wow, that's a popular project then I guess. Well, you're doing great things then, I guess. Yeah, I think it helps a little bit that we come default with Node and most developers hopefully enjoy the work that we're doing. Yeah. Great to see.

So we're going to go answer some questions, if you're up for it. If you want to still ask questions, you can still do it in the Discord channel, of course. We have a question from Ricardo. He's saying three things, actually. He's saying, are Boris as a tree shaking? Nicely found. Or managing the dependence tree? I mean, wow, workspaces are a big win. This was always the argument of using Yarn over NPM. Not anymore. Yay! So he's really happy with workspaces. Yeah, we're pretty excited about providing that for the community. We want to have a best-in-class workspace experience. So, yeah, we're pretty excited about filling out and having more features that are dedicated to those types of use cases for folks that have mono repo projects. Yeah. Awesome.

Sb, you're asking a question to Thomas, but I think you mean Darcy, because he's asking, what are the plans about npm v6? So for npm v6, we've done in the past with major versions, the npm client team has supported security updates to v like previous major releases. So as of right now, the plan is to continue to maintain npm v6, in terms of any security patches that we need to float to that. The hope is to maintain that until the last node LTS version that relies on it goes end of life. So it means we'll be supporting it for for a little bit here and trying to maintain, you know, if there's any CVEs filed against dependencies of v6, we'll do our best to stay on top of those. But the idea is that, you know, v6 is essentially feature complete, and we're hoping people will upgrade to v7, and we'll get a whole bunch of benefits from doing that, including, you know, performance wins. And it's just a much better code base now on v7. So yeah, we're hoping people will jump on board and yeah. Awesome.

And what I'm curious about is, so you as a maintainer of this big project, are you using it yourself? So are you only maintaining it, or is it like you go in the weekends and you build side projects and try to use NPM? Or how do you keep up to date with being a user? So I use a bunch of different tools in a lot of different projects, but I definitely use v7 right now for anything that I'm doing with NPM projects.

So, yeah, I'm definitely a heavy consumer. And because v7 is actually shipping in Node 15 right now, and I like to be on the latest greatest version of Node. It's really easy to stay up to date. I use NPM as a Node version manager. So that's easy for me to use in my projects, the latest version of NPM v7 as well. I don't have many monorepo projects, so I haven't been using workspaces quite as heavily, but we're starting to actually dog food workspaces specifically and there's a lot of cool utilities that we've added into v7.

So NPM diff is something that I've started to use almost every day, especially for like change logs. It's a really nice tool to be able to sort of diff between your last release. And especially for change logs, it's easy to pull that kind of information out of those files. And yeah, I've been using, you know, obviously installing Peer Depths by default is something that I also really rely on now, and I don't have to manage my Peer Depths in my projects manually, which is kind of nice. So yeah, I'm using it every day. So I'm definitely a consumer of the project.

Can you tell us a little bit about, so I happen to know that you've been working on NPM even before it was acquired by GitHub. Can you tell us a little bit about your history there and how it went with acquiring a GitHub and whatever you are allowed to share, of course? I can tell you a little bit. It's actually been a really good acquisition for us. We think that NPM has found a great home with GitHub and also with the Microsoft organization. I joined actually the NPM Inc. team about 10 months before we were acquired and we were working pretty hard to refresh and modernize a bunch of different areas of the company. I think that that was also pretty attractive to GitHub. I think that they saw a good fit there. I think it was just the writing was on the wall that these two organizations should be one. Yeah, it's been really exciting to, behind the scenes, start to collaborate with GitHub and also see what the future roadmap looks like for the kinds of features that we can offer the community based on this tight collaboration. So there's lots of stuff in the works that I can't really speak to right now, but there should be some news come fall time around lots of stuff that we're doing with NPM and GitHub. And obviously, the CLI will continue to improve. And that's also been a huge win for us. Our team has grown, and we can definitely support and maintain the CLI projects into the future here for a long time. Awesome.

And that's also been a huge win for us. Our team has grown, and we can definitely support and maintain the CLI projects into the future here for a long time.

Awesome. Awesome. Let's sidetrack. What's your favorite NPM joke abbreviation? Favorite joke? Yeah. So on the website, you never say Node Package Manager, right? You say NPM, and then it's always something different. What's your favorite? Yeah. I think one's like Ninja Pizza Monsters. So I think it's kind of a reference to like Teenage Mutant Ninja Turtles. So I think that's one one that I've seen. I know that there's thousands of them. That list lives somewhere in GitHub, and folks have added to it over the years. And it's a pretty long list of abbreviations.

Yeah. I know you can just submit a pull request. And I've once scrolled through that list, and it is indeed endless and super funny. And I always like when companies—well, especially big companies—have these kind of jokes and don't take themselves too seriously. That's a fun to see. Yeah, it's pretty interesting. A lot of folks don't know that NPM is not an acronym for Node Package Manager. So we actually have a proper definition on the README now, just so folks know. It's a really interesting history there, and I'll let people go read the history of the actual name NPM. It's there on the CLI's README. Now you're trying to drive some extra traffic. You're stealing all our traffic. I hope your server is ready.

I think we have run out of questions. So if anyone wants to ask Darcy any more questions, Darcy will be available on discord. He's not available on spatial chat. Darcy, thanks a lot for joining us and hopefully we will be seeing you again soon and maybe in real life. That'd be amazing. Thank you so much. Bye bye. Bye. Bye.