DevOps.js

DevOps.js Conf 2022

Levelling up Monorepos with npm Workspaces

Ruy Adorno

JS Nation

JSNation 2022

 Hello, and welcome to leveling up monoripo's with npm workspaces. Let me start by introducing myself. My name is Ria dorno. I'm a software developer in the npm CLI team with GitHub. I have been working on the ends npmcli for almost three years. Now. I worked on the npm V7 review, right and a lot of those features that we're gonna be talking about here today and specially Works places. So I'm super excited to be sharing some of that work with you here today and let's start with an overview what we're gonna be covering here today. Let's start with an intro to what isn't GM Works basis and then cover some very practical examples with some trying to get as close to real life. Usually just possible and also know some of the stuff that you should be looking forward to so let's get started npm workspaces. What is it? So let's start with workspaces, which is basically this concept introducing introduced by yarn a while ago. And basically it is a way to help you manage many packages within a single repo and for npm npm workspaces is kind of the broad name. We gave the set of features that help you achieve that basically help you manage multiple nested package within a single top level package. So it's going to help you centralize the installation of all independencies into US single node modules folder. So you're gonna end up also having only a single package lock file. and there are some advantages to that it is definitely the idea way for managing mono repos and thanks to that. You can have a single place to manage on your issues and basically manage your project and your community. So but also into the technical side, you can also centralize have a single place to run on your tests feel lenting anything that you can kind of imagine you kind of need everything together to get your project running with all the packages. You can have it on a single place. So it might help a lot depending on the style of the project you're trying to achieve. so another thing I would like to note here is a timeline just to give a little bit of this notion of how we've been iteratively improving on npm workspaces and you can see it all started in August 2020 with the release of the really 7.0 of the npmcli that first introduce support to installing npm. Well the workspaces and and then in version 7 that's 7 another big one which kind of first introduced the configuration properties of a workspace or targeting all the works places and the First Command that support them within can run and npm is Zach so There were many more but I just kind of wanted to illustrate how we've been improving and you can definitely keep waiting. If you could definitely wait to see more improvements coming npm workspaces. So let's get started with some examples here. I have this project that I haven't left hand side here. That is a simple app that I put together trying to get as close as possible. For real life project. This is kind of like a web app that consists into different services so you can see I have my user sync service my web app service and I even configure a third workspace here, which is those lights that you're seeing right now. So they're all part of this moment that I'm managing here. And with that I'm hope to put some examples that are real close to how you would use this in a real life so you can see there they all have the words of dependence is here using next JS the services using fastify so I can just quickly start by running in pmls here and we can see how and LS is a command that is aware of workspaces and it's going to highlight each of the workspaces and my project and even list the dependencies of each workspace when I run in MLS. so it's a first command to note that has first class support to workspaces So moving forward here. I'd like to highlight npm in it. It's probably the best way of just start a new workspace because it's going to make sure every step that is needed is going to end up being there. So basically everything you need to track a nested package as a workspace is make sure you have that the folder with a package Jason inside it inside your project. And then just add that folder name to your workspaces field of your package Jason in the top level. So using npm in it is going to make sure that those requirements are there so gonna run a quick example here in my sample app. I'm gonna create a new workspace. Let's say I'm gonna call it the website so I can run npm in it. I'm using Dash Why here to just accept all the defaults and targeting a website. So that's going to create a website folder with a package Jason file inside it and it's going to print the contents of the package Jason file over here. And as I mentioned before the other, we really important point that that npm in it takes care of is placing the reference to the website inside my package Jason file in my top level folder. So with that. It is all set up if I npm install here. Now, the install tree is going to be tracking the website and if I run npmls again, I'm gonna see website listed as one of the workspaces here highlighted in green, so That is definitely the recommended way to get a started with a new workspace inside your project. And we can move on to another very important aspect. How do you add dependencies to one of your workspaces? Right? So A real quick example here again. I'm gonna shift around the country workspace configuration here. You can so you can see that you can practically go anywhere so I can just declare npm workspace. I want to Target the website. I just created let's say I want to install a package. I'm gonna install this simple slider package into my website, so I can run that command. So what's going to do what's going to install the package name simple slider from the registry. It's going to put in the as it dependency of my website workspace so I can look up its package Json. And I can see that the samples lighter is declared here as a dependency. But if I look up, oh my the node modules folder inside the top level of my mono repo app. I can see that simple slider was put there so that's basically the one of the most important concepts with the npm workspaces is that everything is trying to be hoisted as much as it can it only avoids that hoisting or or that bringing one level up each dependency if it finds duplications. So if there are duplications, then you might end up with a node modules folder inside your workplace that did duplicate those packets so that it can work as expected. So great all that to mention one thing that I've seen as a common repeating. Thing with workspaces is that some people use to have there's no hoist configuration option. That was an option that yarn introduced in order to basically have workspaces say, okay. I need this dependency, but I need to make sure it is inside a note modules folder. Inside my workspace. So because then because of the nature of some module some packets, they really need to rely on the fact that they are that position the file system. So that's something that you are come up with in order to basically just give a scape hats to users to kind of have a little bit more control over where they it's placing their dependencies. So For npm workspaces. We are currently tracking that as a new feature. We're discussing it in your RFC process this and like We are looking forward to provide users with a way of doing it. We're just not sure yet. If it is going to be using that same no highest config failure or it's gonna be something different. But meanwhile, I wanted to basically provide a way to unblock yourself, if you ever hit that scenario and my sample app here. I actually hit it. I am using this Prince Michael client, which is the library to manage. Basically. It's an orm for that basis and I'm using it in both my services here. of this sample app and when it got hoisted to the top of my app, it basically didn't work. So a quick way of unblocking me and making it work is to duplicate it and have it being in both of my services folder inside it there no modules. So one quick way to achieve that duplication was basically that I put some mot package here that basically would Conflict with the namespace of the package there they're using so I've used prism and Prima client and declare them using npmis and pointing to just something else in my case here. I'm using this abrap module, but it can be anything ready. And so this is in order to make sure that when I declare in like my user saying service here, I have it inside my package Jason. I just have the correct Declaration of the expected versions I have for these libraries so so that when npm installs it is going to be actually putting The Prisma client folders inside my user sync node modules. So it's basically a way of forcing that same behavior of no hoist and it might help unblock other users there. So it's a very practical way of get you going today, but definitely keep an eye because there is definitely Things going on it might improve very soon in the default and pmcli support. so moving forward what it's also cover running Scripts. and here I also place a few examples of how you can run it. You can run it by where by using the workspace name also an example if your workspace has a scope then you also have to Define scope, but you can also call it using the workspace path name. And basically here I in my project here. I can show real quick if I Target that user sync service workspace that I have and I run its tests. You can see it's going to run tests for that workspace. And it's basically the same thing as navigating to that workspace folder and running and PM test inside it. Yeah, and I want to basically move forward here to more a little bit more complex examples that I put together. basically showing how in a more complex mono repo you can orchestrate your scripts in a way that they can become much more useful. So starting real quick here following up from the task example. I just gave this is the this is an example. This is our descript from my top level package Jason and I'm declaring this test column web app and that's color user saying name after each of my workspace is so and then basically calling the tasks and pointing to each of the workspaces there. So this way I have this multiple scripts in my top level command and I'm gonna be using I'm going to be using this round Dash ass binary file that it actually comes from this user land package name npm run all so I know that it up here. It's a very handy package helps you kind of run multiple scripts and gives you much more flexibility on the way you do it. So basically here I'm telling run ass to run all of my test column targets Define here on my file. So that's jump back to my Root folder here and running PM test and see what happens so you can see it runs the test web app Target first, which is running the test inside that workspace. And then after that you ran the test user saying which ran the test for that usersync works place and you can see succeeded. so a little bit more evolve but kind of following up from this first example, let's run also, my dad web servers here. So I have kind of similar thing that I've called a web app and user saying and I'm pointing in PM run to my workspace is and I'm running the dad script of each one of them but the particularity of the servers that I need all of them running at the same time, right? So this is also something provided by we're in PM right now, which I'm going to be using to run using run Dash p which is going to be running both of these Targets in parallel. It's going to just start them all at the same time keep them running so that I can have everything Ryan running in actually use my the development server navigate to my UI. So let's try it real quick here. I'm gonna run and run Dev. You can see it's gonna run all of the all the scripts and they're dependencies. So going to advance here on my this next slide is actually pointing to the server running on my localhost. And so now once everything puts up I can see my user list is back up again. I can see some users. I can click around navigate and I can see that my web app is working as expected. So here I kind of want to highlight this new command somewhat new. We kind of introduced it enough last year in version 7.20 and it's a command to help you set and retrieve. keys values from your package Jason files, so it is not only useful in the context of workspaces. But it's super useful when it's used in the content of world spaces so you can it can be used to let me highlight here real quick if you run like npm pkgat. It can be just a simple package. It's going to basically print out the content of package Jason and you can also filter out by properties. Let's say name and then I get the name of my package there. So and when you when you bring on the support to the workspaces properties, it's It can actually get really powerful like index example here. I'm going to retrieve the name of the version of all of my workspaces. So this config option here dash dash WS it's basically a way of saying okay Target all of my configured workspaces. So if I run that you can see it return the name of the version of each of my workspaces and they're even Key by the workspace name. so it can be really useful. And also to highlight a little bit more of how they can use. So let me sat also some some data into these packages of files. So let's say I'm managing this project and it's an open source project. I I just want to surface info about how users can found my work so I can go ahead and use npm PKG set. Let's say I'm gonna set a farming key to all of this packages and I'm gonna point it to my GitHub sponsor. URL and I'm going to Target all of the configure workspaces. So again run that and if you look up let's say my user saying back as Jason you can see the funding info is there now so same thing goes to my web app Jason. So you can be an incredible powerful way of just help you manage all of that data in your package. Jason across all of the configured Works basis. so also highlight here that npm version npm publish also have Support workspaces, so if you want to cut a let's say a new patch version of a workspace. That's possible today. And something to keep in mind is you guys can see here. I bump my version to v1.0.1 I can look up. my package Json there. And see that the version is there but one thing to keep in mind that has a bit different now the from the way I'm conversion works by default is that is not all the committing and tagging when it's running in PM version. So it's something to keep in mind, but you can definitely be looking forward to improvements to npm versions specifically. so to kind of wrap up everything I wanted to run a quick example here. I'm using npms ACT and benazak. It's basically npx. It kind of got promoted into subcommanded Ben PM CLI since version 7 and I wanted to kind of like create real quick another workspace. Let's call it the print current working directory and I'm going to be using a Zach to basically round this workspace as a command across all of the other workspaces. I have configured. Just kind of putting everything together using all the commands that we just saw. So let me jump real quick. and to that folder and start index.js file and this is going to just basically print the current work in directory. This is gonna be useful because I want to highlight the nature of how one PMS act or npm run they're running in the context of each folder. So I want to run I want to print the current work and directly just to make sure of that. So there next time I'm gonna npm PKG set. I've been value. Of index.js to that. Just created printers CWD workspace. So that it probably tracked that binary. Index.js and now go to npm install to put everything together. Make sure my been as properly linked to my node modules folder and now I can npm as that. And I'm gonna be calling my module and I'm gonna tell it to run inside each of the workspaces. And you can see I got the expected result here it basically printed the path for each one of my workspaces when it had executed inside them. So just a quick example here. To kind of close wrap up and put everything together how you can use all of these commands. So if you want to learn more about it, definitely look up the official docs. We have a session exclusive on the workspaces, but there is also documentation across each of these commands explaining a little bit more how to use them. And if you want to find me really adorno Twitter or GitHub, and here's also my website, so I think this I really hope this was useful and very good real life examples, and I hope you enjoy. Thank you.

#npm