What's New in npm?

Rate this content
Bookmark

The npm CLI has been, & continues to be, a core developer tool of the Node.js/JavaScript ecosystem. This past year npm@7 became Generally Availble, introducing with it a wealth of changes & net-new capabilities including: lockfile upgrades, workspace support, installing peer dependencies default, npm diff, npm explain, npm exec, npm set-script & much, much more. We'll dive into this work as well as share some exciting news about what to expect in the weeks & months to come.

26 min
01 Jul, 2021

Video Summary and Transcription

Welcome to my talk on what's new in the NPM CLI. NPMv7 introduced many new capabilities, including installing peer dependencies by default. npm v7 also introduced support for workspaces, allowing the definition of projects within your root project. The NPM team is continuously improving the CLI with weekly releases and is working on exciting features in collaboration with GitHub. NPM is not an acronym for Node Package Manager, and the CLI will continue to improve with the support of the growing team.

Available in Español

1. Introduction to NPM CLI

Short description:

Welcome to my talk on what's new in the NPM CLI. NPM is known for installing packages, with almost 125 billion monthly package downloads. But we also support discovery, building, bundling, testing, and more. Our team has grown, with new members and nearly 700 contributors. NPMv7 introduced many new capabilities, including installing peer dependencies by default.

Welcome, everybody. Thank you for joining me today. My name is Darcy Clark, and I'm going to be talking about what's new in the NPM CLI. I want to give a big shout out and that's been running the conference so far and all the folks behind the scenes at DevOpsJS 2021.

A quick overview on who I am. I'm the engineer manager for the NPM CLI team at GitHub. I'm based here in Toronto, Ontario, Canada, and there's a bunch of other places that you can find me on the interwebs.

So, I want to start with talking a little bit about what NPM does or what people think we do. And I think the biggest thing we're known for is installing a whole bunch of packages, so this is our average package download count. I think last time I checked, we're almost at 125 billion package downloads monthly as a community, which is a pretty amazing mark. I think we had about 100 billion average monthly downloads back in September, October, October, around the 11th anniversary of the 11th year of NPM, which is pretty cool. We do more than just install packages and publish packages. So the NPM CLI supports also discovery, building and bundling your packages, testing, and a whole bunch of other features with a number of commands. And if you're looking for information on the 61 plus commands and growing, you can go check out our docs at docs.npmjs.com. You'll notice that they just recently got a nice new design, a refresh, and there's a ton of information about the config as well as the commands that we support.

So what has changed in the NPM CLI? Because you might be wondering, I switched tooling a while back. What has changed or what's new in the NPM CLI? Well, what's changed quite a bit is our team. Over the last couple of years, we've added and had new folks join our team. Currently we have a core team of myself, Roy Adorno, Nathan Lafreniere, and Michael Garvan, our GAR. We're also expanding this team and adding three new folks come next month. And we also get a ton of contributors, almost 700 contributors to the NPM CLI, which is pretty amazing. What's also changed is that we just recently shipped NPMv7 and went latest or sort of generally available in January 2021. So, in v7 we introduced a lot of new capabilities, including some improvements to legacy commands and also added things like workspace support along with much, much more. So, let's dive into some of these changes. So as a v7, one of the biggest things that you'll note is that we've started to install peer dependencies by default. Peer dependencies are not new by any stretch of the imagination, they've been around for about eight years. But what we noticed is that a lot of projects were starting to have to, and a lot of developers are starting to have to manually manage these dependencies, which is a problem. NPM is a package manager and we want to manage your packages. So in this example, you can see that I have two workspace projects that actually depend on different versions of react. These are essentially conflicting versions, and we don't quite know what we...

2. New Features and Improvements in npm v7

Short description:

Previously, in v7, we now install and resolve peer dependencies, providing warnings and actionable steps if we can't. The logic for this resolution has been moved to a new package called Arborist. We also introduced npm explain, which shows why a package was installed. npm exec now asks for confirmation before installing new packages. npm audit has been refactored for easier vulnerability management. npm diff allows you to see the difference between package versions. npm v7 introduced a new package lock schema and improved performance.

Previously, we would just leave these be and let the developers essentially manage these. As a v7, we will now try to do our best to install and resolve your peer dependencies. And if we can't, we'll print a sort of a warning with some actionable steps for you to take to essentially resolve these dependencies. And if you're looking for the actual logic that is running and doing this resolution, we've moved everything that we... Sort of the brains of the operations into a net new package that we call Arborist.

So moving on, we actually introduced, to sort of help with this change, we also introduced npm explain, or the alias npmy. And this command actually functions a lot similar to yarn's npmy and npm explain, where it will actually show you the reason why a package was installed. So in this case, I've run npm explain chalk, and asking essentially, why is chalk installed, and npm will let you know that in this case, I have a workspace project called b, and it actually includes chalk as a peer dependency, and that's the reason why that got installed. So some nice developer, you know, experience and developer tooling to sort of help with ergonomics around managing your dependencies.

Also, what's kind of new npm exec was introduced in v7. It's essentially the guts of npx. So if you've ever used npx, then this will be, you know, not very new to you. We added some safeguards with npm exec and npx. In v7, we now actually ask before installing a package we've never seen before, we actually ask you to confirm and have a confirmation prompt to ensure that you don't accidentally install something and run something that you didn't mean to.

So also as of v7, we've made significant changes to npm audit. npm audit got sort of a an entire refactor in terms of the logic, performance, and also ui. So now we hope that it's a lot easier for developers to quickly grasp what exactly is going on and what the flags are for the different dependencies that might have a vulnerability issued against them. And we hope that this is actually a way faster experience for developers and we think it's an improved experience overall for sure. And a net new command that we actually introduced just recently is npm diff. With npm diff, it works very similar to get diff in that you can actually specify a package name, a version or a file and you can essentially see the difference between those two things. This is great. In this example I'm essentially seeing what's changed between two different versions of my package called sleepover and I can see that the version was updated and the package json was changed slightly.

Another big change in npm v7 was the introduction of a new package lock schema. Previously we were on v1 of the package lock and as you transition or as you upgrade to npm v7 we will seamlessly upgrade your version of your package lock file to meet essentially this new schema. So there shouldn't be any problems with upgrading and if you have any problems with developers using v6 or v7 you can always say no save or provide the flag no save to not update your package lock files. So another huge win that we sort of accomplished with npm v7 was some improvements to the actual performance of a lot of the commands. Specifically on install we initially saw a sort of a degradation but we noted that we were actually installing more dependencies because of the changes in behavior that we had introduced with peer dependencies. So peer dependency resolution bumped out some of our install times because we're actually installing and resolving more dependencies so this was somewhat expected, and as you can see we've started to benchmark across different sort of scenarios and fixtures including installing peer depths. So that's that last benchmark there at the bottom. As we sort of move forward we actually saw that we made actual significant improvements against ourselves and are hoping to continue this work inside of the benchmark suites that we're running and are hoping to continue to focus on performance as a place for us to optimize and get better.

3. Support for Workspaces in NPM v7

Short description:

npm v7 introduced support for workspaces, allowing the definition of projects within your root project. The work has been divided into two phases, with the second phase focusing on making more commands workspace aware. Two new configurations, 'workspaces' and 'workspace', will be released, allowing you to run commands against defined workspaces. This new capability is available in NPM v7.7, with further support coming in future releases.

And so if you're ever interested in the work we're doing there you can go to npm slash benchmarks and you can go see the different runs that we're doing against the different versions of npm that we're building.

So npm v7 introduced support for workspaces. This was a huge win for our team. It's something that we wanted to offer to the developer community for a while now. And the basic support here for workspaces is just the definition of the projects within your root project.

So in this case, in this example, I've just defined essentially an A and a B workspace. And as of v7, 7.0.0, the only real command that understood or supported these workspace definitions was npm install. And I actually have a reference there to our docs. And there's references also back to the RFC that we originally created for this basic support.

So since then, we sort of broke out the work into another two phases. And the second of which is what we've been working on just recently, which is making more commands essentially workspace aware. And this second stage of this work, really encompasses all the other 60 plus commands that we have to define them and categorize them in terms of their workspace awareness.

So what you'll see is two new configurations that are actually going to be released in the next week or so, along with some other work that is associated with workspace awareness. So the first config that we have is workspaces. And it's essentially a bullion to be passed to any of the NPM commands to let you know that you'd like to run that command against the workspaces that you've defined. And then the second config is sort of a filtered command for the specific workspace you'd like to run the command against. So that's just workspaces and workspace. So ws and w are the aliases.

And if we see what this looks like in practice, so as of NPM 7.7, you will be able to run on any of the lifecycle scripts or lifecycle commands like run scripts, run test, start, stop, restart with the ws or w flags. In this case, I've run test with ws, and it goes and runs all the test scripts that are in the workspaces that I've defined in my project. And here, in this case, in this example, I've actually filtered just to run the test for the workspace A that I've defined in this project.

And workspace is pretty nice because it's very composable. You can actually define workspace many times. So, in this case, in this case, I've actually defined B and then A. And because of the way that this is implemented, they run serially. And you can see that B's tests run first and then A's. And if I were to define more and more workspaces, you'd essentially see more and more runs done.

So, this new feature, this new capability with working with workspaces is available as of NPM v7.7, which we're releasing in the next few days. 7.8 will have exec and init support and the rest of the subcommands becoming workspace-aware will come subsequently after that. If you're ever looking to get the latest version of NPM, you can install us with npmi npm-g.

4. NPM Pipeline and Future Plans

Short description:

We ship weekly now, with new patches, miners, and improvements coming each week. Our pipeline includes upcoming features like registry protocol and package overrides, as well as workspace improvements and filtered/scoped installs. If you want to learn more, visit the npm-RFCs forum or join our weekly npm-OpenRFC call. Thank you to everyone involved in the event!

We ship weekly now. You'll be seeing new patches, new miners, new improvements coming each week. I want to take some time not just to talk about what we have been shipping and what we are going to be shipping, but what does our pipeline look like, what's coming down the road.

In the quarters to come, the Q2 and Q3 of 2020, we are looking at registry protocol and package overrides as two RFCs that are in our back pocket that we want to essentially queue up the work for, including published prompts in Q3, which is another RFC that has been approved, and we just need to do the work. As noted earlier, we are doing workspace improvements right now, and hoping to finish off the quarter with also filtered or scoped installs, which is a really cool feature.

If you'd like to learn more about the registry protocol or the override specs and RFCs that are being proposed right now, you can go jump over into the npm-RFCs forum and go check out what those look like. And if you want to get involved even beyond that, again we run a weekly npm-OpenRFC call, which happens at Wednesdays at 2 p.m. Eastern, and that's a live call where we talk about the future, current and future of the project and discuss essentially future requests and things that we would like to get done. We also have resources for the docs that I referenced earlier, our bug tracker, events that we're at, that we're going to be at, or speaking at, our roadmap and other feedback resources at npm.community.

And that's it for me today. So just wanted to give you a big thanks for everybody that's been involved in the event and feel free to reach out at one of these links here below. Cheers.

QnA

Discussion and Q&A

Short description:

Good to see you! We have a question from Ricardo who is really happy with workspaces. We're excited about providing a best-in-class workspace experience. As for npm v6, we'll continue to maintain it for security patches, but we encourage upgrading to v7 for performance wins and a better code base. As a maintainer, I use v7 for my NPM projects.

Hey, good to see you. So we're going to look at the results of your question. Have you used the npm-cli previously? And 100% percent saying yes, wow, that's a popular project then I guess. Well, you're doing great things then, I guess. Yeah, I think it helps a little bit that we come default with Node and most developers hopefully enjoy the work that we're doing. Yeah. Great to see.

So we're going to go answer some questions, if you're up for it. If you want to still ask questions, you can still do it in the Discord channel, of course. We have a question from Ricardo. He's saying three things, actually. He's saying, are Boris as a tree shaking? Nicely found. Or managing the dependence tree? I mean, wow, workspaces are a big win. This was always the argument of using Yarn over NPM. Not anymore. Yay! So he's really happy with workspaces. Yeah, we're pretty excited about providing that for the community. We want to have a best-in-class workspace experience. So, yeah, we're pretty excited about filling out and having more features that are dedicated to those types of use cases for folks that have mono repo projects. Yeah. Awesome.

Sb, you're asking a question to Thomas, but I think you mean Darcy, because he's asking, what are the plans about npm v6? So for npm v6, we've done in the past with major versions, the npm client team has supported security updates to v like previous major releases. So as of right now, the plan is to continue to maintain npm v6, in terms of any security patches that we need to float to that. The hope is to maintain that until the last node LTS version that relies on it goes end of life. So it means we'll be supporting it for for a little bit here and trying to maintain, you know, if there's any CVEs filed against dependencies of v6, we'll do our best to stay on top of those. But the idea is that, you know, v6 is essentially feature complete, and we're hoping people will upgrade to v7, and we'll get a whole bunch of benefits from doing that, including, you know, performance wins. And it's just a much better code base now on v7. So yeah, we're hoping people will jump on board and yeah. Awesome.

And what I'm curious about is, so you as a maintainer of this big project, are you using it yourself? So are you only maintaining it, or is it like you go in the weekends and you build side projects and try to use NPM? Or how do you keep up to date with being a user? So I use a bunch of different tools in a lot of different projects, but I definitely use v7 right now for anything that I'm doing with NPM projects.

NPM Usage and Acquisition by GitHub

Short description:

I'm a heavy consumer of NPM and always stay up to date with the latest version. Using NPM as a Node version manager makes it easy to use the latest NPM v7 in my projects. While I haven't used workspaces extensively, I've started exploring the utilities in v7, such as NPM diff and automatic installation of Peer Dependencies. The acquisition of NPM by GitHub and Microsoft has been a positive move, allowing for collaboration and exciting future features. We're working on many things with NPM and GitHub, and the CLI will continue to improve with the support of our growing team.

So, yeah, I'm definitely a heavy consumer. And because v7 is actually shipping in Node 15 right now, and I like to be on the latest greatest version of Node. It's really easy to stay up to date. I use NPM as a Node version manager. So that's easy for me to use in my projects, the latest version of NPM v7 as well. I don't have many monorepo projects, so I haven't been using workspaces quite as heavily, but we're starting to actually dog food workspaces specifically and there's a lot of cool utilities that we've added into v7.

So NPM diff is something that I've started to use almost every day, especially for like change logs. It's a really nice tool to be able to sort of diff between your last release. And especially for change logs, it's easy to pull that kind of information out of those files. And yeah, I've been using, you know, obviously installing Peer Depths by default is something that I also really rely on now, and I don't have to manage my Peer Depths in my projects manually, which is kind of nice. So yeah, I'm using it every day. So I'm definitely a consumer of the project.

Can you tell us a little bit about, so I happen to know that you've been working on NPM even before it was acquired by GitHub. Can you tell us a little bit about your history there and how it went with acquiring a GitHub and whatever you are allowed to share, of course? I can tell you a little bit. It's actually been a really good acquisition for us. We think that NPM has found a great home with GitHub and also with the Microsoft organization. I joined actually the NPM Inc. team about 10 months before we were acquired and we were working pretty hard to refresh and modernize a bunch of different areas of the company. I think that that was also pretty attractive to GitHub. I think that they saw a good fit there. I think it was just the writing was on the wall that these two organizations should be one. Yeah, it's been really exciting to, behind the scenes, start to collaborate with GitHub and also see what the future roadmap looks like for the kinds of features that we can offer the community based on this tight collaboration. So there's lots of stuff in the works that I can't really speak to right now, but there should be some news come fall time around lots of stuff that we're doing with NPM and GitHub. And obviously, the CLI will continue to improve. And that's also been a huge win for us. Our team has grown, and we can definitely support and maintain the CLI projects into the future here for a long time. Awesome.

NPM Joke Abbreviations and Conclusion

Short description:

Our team has grown, and we can definitely support and maintain the CLI projects into the future. NPM has a list of thousands of funny joke abbreviations. A lot of folks don't know that NPM is not an acronym for Node Package Manager. Darcy will be available on discord for further questions. Thank you and goodbye!

And that's also been a huge win for us. Our team has grown, and we can definitely support and maintain the CLI projects into the future here for a long time.

Awesome. Awesome. Let's sidetrack. What's your favorite NPM joke abbreviation? Favorite joke? Yeah. So on the website, you never say Node Package Manager, right? You say NPM, and then it's always something different. What's your favorite? Yeah. I think one's like Ninja Pizza Monsters. So I think it's kind of a reference to like Teenage Mutant Ninja Turtles. So I think that's one one that I've seen. I know that there's thousands of them. That list lives somewhere in GitHub, and folks have added to it over the years. And it's a pretty long list of abbreviations.

Yeah. I know you can just submit a pull request. And I've once scrolled through that list, and it is indeed endless and super funny. And I always like when companies—well, especially big companies—have these kind of jokes and don't take themselves too seriously. That's a fun to see. Yeah, it's pretty interesting. A lot of folks don't know that NPM is not an acronym for Node Package Manager. So we actually have a proper definition on the README now, just so folks know. It's a really interesting history there, and I'll let people go read the history of the actual name NPM. It's there on the CLI's README. Now you're trying to drive some extra traffic. You're stealing all our traffic. I hope your server is ready.

I think we have run out of questions. So if anyone wants to ask Darcy any more questions, Darcy will be available on discord. He's not available on spatial chat. Darcy, thanks a lot for joining us and hopefully we will be seeing you again soon and maybe in real life. That'd be amazing. Thank you so much. Bye bye. Bye. Bye.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Remix Conf Europe 2022Remix Conf Europe 2022
23 min
Scaling Up with Remix and Micro Frontends
Top Content
Do you have a large product built by many teams? Are you struggling to release often? Did your frontend turn into a massive unmaintainable monolith? If, like me, you’ve answered yes to any of those questions, this talk is for you! I’ll show you exactly how you can build a micro frontend architecture with Remix to solve those challenges.
Remix Conf Europe 2022Remix Conf Europe 2022
37 min
Full Stack Components
Top Content
Remix is a web framework that gives you the simple mental model of a Multi-Page App (MPA) but the power and capabilities of a Single-Page App (SPA). One of the big challenges of SPAs is network management resulting in a great deal of indirection and buggy code. This is especially noticeable in application state which Remix completely eliminates, but it's also an issue in individual components that communicate with a single-purpose backend endpoint (like a combobox search for example).
In this talk, Kent will demonstrate how Remix enables you to build complex UI components that are connected to a backend in the simplest and most powerful way you've ever seen. Leaving you time to chill with your family or whatever else you do for fun.
JSNation Live 2021JSNation Live 2021
29 min
Making JavaScript on WebAssembly Fast
Top Content
JavaScript in the browser runs many times faster than it did two decades ago. And that happened because the browser vendors spent that time working on intensive performance optimizations in their JavaScript engines.Because of this optimization work, JavaScript is now running in many places besides the browser. But there are still some environments where the JS engines can’t apply those optimizations in the right way to make things fast.We’re working to solve this, beginning a whole new wave of JavaScript optimization work. We’re improving JavaScript performance for entirely different environments, where different rules apply. And this is possible because of WebAssembly. In this talk, I'll explain how this all works and what's coming next.
React Summit 2023React Summit 2023
24 min
Debugging JS
As developers, we spend much of our time debugging apps - often code we didn't even write. Sadly, few developers have ever been taught how to approach debugging - it's something most of us learn through painful experience.  The good news is you _can_ learn how to debug effectively, and there's several key techniques and tools you can use for debugging JS and React apps.
Node Congress 2022Node Congress 2022
26 min
It's a Jungle Out There: What's Really Going on Inside Your Node_Modules Folder
Top Content
Do you know what’s really going on in your node_modules folder? Software supply chain attacks have exploded over the past 12 months and they’re only accelerating in 2022 and beyond. We’ll dive into examples of recent supply chain attacks and what concrete steps you can take to protect your team from this emerging threat.
You can check the slides for Feross' talk here.

Workshops on related topic

React Day Berlin 2022React Day Berlin 2022
86 min
Using CodeMirror to Build a JavaScript Editor with Linting and AutoComplete
Top Content
WorkshopFree
Using a library might seem easy at first glance, but how do you choose the right library? How do you upgrade an existing one? And how do you wade through the documentation to find what you want?
In this workshop, we’ll discuss all these finer points while going through a general example of building a code editor using CodeMirror in React. All while sharing some of the nuances our team learned about using this library and some problems we encountered.
Node Congress 2023Node Congress 2023
109 min
Node.js Masterclass
Workshop
Have you ever struggled with designing and structuring your Node.js applications? Building applications that are well organised, testable and extendable is not always easy. It can often turn out to be a lot more complicated than you expect it to be. In this live event Matteo will show you how he builds Node.js applications from scratch. You’ll learn how he approaches application design, and the philosophies that he applies to create modular, maintainable and effective applications.

Level: intermediate
TestJS Summit - January, 2021TestJS Summit - January, 2021
173 min
Testing Web Applications Using Cypress
WorkshopFree
This workshop will teach you the basics of writing useful end-to-end tests using Cypress Test Runner.
We will cover writing tests, covering every application feature, structuring tests, intercepting network requests, and setting up the backend data.
Anyone who knows JavaScript programming language and has NPM installed would be able to follow along.
Node Congress 2023Node Congress 2023
63 min
0 to Auth in an Hour Using NodeJS SDK
WorkshopFree
Passwordless authentication may seem complex, but it is simple to add it to any app using the right tool.
We will enhance a full-stack JS application (Node.JS backend + React frontend) to authenticate users with OAuth (social login) and One Time Passwords (email), including:- User authentication - Managing user interactions, returning session / refresh JWTs- Session management and validation - Storing the session for subsequent client requests, validating / refreshing sessions
At the end of the workshop, we will also touch on another approach to code authentication using frontend Descope Flows (drag-and-drop workflows), while keeping only session validation in the backend. With this, we will also show how easy it is to enable biometrics and other passwordless authentication methods.
Table of contents- A quick intro to core authentication concepts- Coding- Why passwordless matters
Prerequisites- IDE for your choice- Node 18 or higher
JSNation 2023JSNation 2023
104 min
Build and Deploy a Backend With Fastify & Platformatic
WorkshopFree
Platformatic allows you to rapidly develop GraphQL and REST APIs with minimal effort. The best part is that it also allows you to unleash the full potential of Node.js and Fastify whenever you need to. You can fully customise a Platformatic application by writing your own additional features and plugins. In the workshop, we’ll cover both our Open Source modules and our Cloud offering:- Platformatic OSS (open-source software) — Tools and libraries for rapidly building robust applications with Node.js (https://oss.platformatic.dev/).- Platformatic Cloud (currently in beta) — Our hosting platform that includes features such as preview apps, built-in metrics and integration with your Git flow (https://platformatic.dev/). 
In this workshop you'll learn how to develop APIs with Fastify and deploy them to the Platformatic Cloud.
JSNation Live 2021JSNation Live 2021
156 min
Building a Hyper Fast Web Server with Deno
WorkshopFree
Deno 1.9 introduced a new web server API that takes advantage of Hyper, a fast and correct HTTP implementation for Rust. Using this API instead of the std/http implementation increases performance and provides support for HTTP2. In this workshop, learn how to create a web server utilizing Hyper under the hood and boost the performance for your web apps.