Remixing Your Stack in a Monorepo Workspace

Hey, let's talk about how we can remix our stack in a monorepo workspace. And I would like to start with a quote, or actually a tweet from Ryan Florence from a couple of weeks ago, where he actually made a quite powerful statement and mentions that your routing files should actually just configure the routing for your specific stack. The actual implementation of that feature that's being visualized there should live somewhere else, right? So you import it, but then specifically, depending on the stack you have, you configure routing and you import it and then it gets bundled and visualized on the webpage. And this is a very powerful mechanism for architecting your system, but also for organizing your code. And we can take this even further. So normally we see exactly this type of structure, what Ryan Florence mentioned. So you have kind of your application, you have the routing folder, which can be pages, routes, app, whatever. And then you should have, as Ryan mentions, like a different folder potentially inside that app, which is like your features. And then you have like the different folders for all those features. And the routing simply imports from them. Now we could actually go a step further and even move them out of the application into a dedicated packages folder. So this new structure, what you see now is we have an apps folder and now we have a packages folder and in the apps folder, we still have our application with its routing mechanism. And then in the packages folder, we now have different folders for all of our features. And all of these features are now nicely kind of independent, encapsulated, and they have a very clean API point where there is an index.ts file, which exposes everything that can be reused outside and everything else kind of remains inside that package. So you can imagine we get a very clean structure having such a setup here. Obviously, we can go ahead and just import as before. Our import might even look like as if we consume it from some external package, like some MPM registry, but it actually is just consumed locally in this case. But you can see it here from such an import statement. So how does this help actually? Well potentially with something like incremental migration. Now you might have noticed that in my example, I used a Next.js app, right? Just that pages folder. Well, if you use the new features or opt into new features, you might have different routing folder there. But this was not by accident, but it was intentional. So we might nowadays have like a Next.js application with some already built-in features, with some like an application that has been developed already for a couple of months or even years. And you might want to kind of slowly move over to a Remix stack because for whatever reason that is more suitable for you. So this is exactly what you could achieve in such a setup because as you can see, our actual Next.js app and the actual feature folders are kind of already separated nicely. And if you pay attention, this is actually already a monorepo if you want, right? We have just a single app in there, but we have an app, we have a couple of packages, so this is totally a monorepo, a very simple one. So what we could do is we could just add another application. In that case, for instance, add a new Remix app, which is our target migration destination, right? So we want to move over new features or maybe we even want to keep them both depending on the use case or deployment strategy. So we could just generate a new Remix app with the create-remix command and then just add that app inside our monorepo workspace and then import the features that we already have. There might be some various feature specific things or even things that are coupled to the Next.js application obviously. So those need or might need some refactoring, but it's much easier because they have already been moved out. So how does this work in practice? Let me show you. But first, my name is Juergen Strompflohner. I'm a Google Developer Expert in web technologies. I'm also an AgCAD instructor and I work as the Director of Developer Experience for a tool called NX. And NX is a smart, fast, extensible build system that happens to work really well for monorepo setups and application structures that we have just discussed. So let's have a look. So I have here my Next.js app with that features folder, which gets imported directly from our routing configuration. And those features are actually nicely organized. So there's an index.js file which exports our functionality, which is super simple. And then we have also the design system which exports here a single button. But this is organized in the same fashion. So how would we refactor this to get such a monorepo workspace with a more modular structure? Well, first of all, we would probably go ahead and create the folders. And by the way, these are fully up to you, however you name these. The normal naming is what you find out there is like apps and packages or apps and libs. But it's up to you how you want to choose those. And we would probably go ahead and move our next app in this case into this app folder. As a next step, we need to factor out these features and move them to their packages folder. So they are completely independent. And as a result, we should also now make all of these autonomous in the sense that they have their own package.json, their own dependencies, their own build scripts, and things like that. So let me set that up for you quickly. So here we go. We now have a source folder where the files live. We have a neos build config here, which is a very simple one. Each of these packages has now its package.json with the name build script in this case, and its own dev dependencies. And the very same setup holds for that design system package as well. Now the next step is we need to actually configure that monorepo to tell npm that whenever we reference a package, it should reference it locally rather than globally. So to do that, we need a root level package.json here. So that package.json is the configuration of our monorepo. So there is most importantly that workspaces property, which defines what our local folders are where packages should be linked between each other. It might also have some global dev dependencies as well as dependencies that all of the packages in the workspace should share among them. But as a next step, we can go ahead and install all those dependencies at the very root of the package. So with this, we should now be able to actually link here our packages to our next.js application, because obviously our previous import, as you have seen here from the local features folder doesn't work anymore because those features don't exist anymore. So we should now go ahead and actually link them to where they are now moved. And so in this case, I would go here for the authentication feature and import it from here my org auth, as well as here for this one, I would have here my org design system. But we also need to reference those dependencies in our next.js app. So we would go here in the package.json of our next.js app in the dependencies folder and reference here my org design system. We can reference it with a star because we're not really interested in a specific version, although we could, but we rather want to consume the latest version that lives locally here in our workspace. And we do the very same for our auth part. Now let's install the dependencies again so npm can do the proper linking. But before we can then actually consume them, we also need to build these packages because obviously those depend on the build output of compiling this to a JavaScript file. So let's go ahead and build those. So we can run npm run build and now we can give it the workspace flag here and say my org auth, which should now go ahead and build our authentication library. And you can see here it produced a dist folder, it produced like the index DTS files. And so we can do the same thing here also for our design system library. So we again run the build. And we similarly will get here a dist folder with the compiled files. So if you now go ahead and run our next JS application, it would totally work. You can see here we get the name printed out as well as our small button visualized on the web page. So you might have noticed that there are some things in the setup which are not ideal, especially in terms of the DX. Because for instance, whenever we run our next JS application, we need to make always sure that there is a build already run for our auth and design system packages. So we need to have that dist folder already there because our next app depends on those build outputs. And similarly also if we change something in those packages folders, there needs to be some watching process that kind of refreshes the actual application. Now adding an X for instance can help with some of these things. So let's do that. Adding an X is pretty straightforward. So you run the npx nx add latest command and you run the init script. And what this will do is it will add the nx package to this monorepo workspace. And then it will ask you already a couple of questions to configure it. First of all, the first question is exactly what I mentioned before. So is there any, are there any scripts that need to be run in order? And our build script, as we have just mentioned, definitely needs to run in order because we need to run the dependencies first. Similarly, nx has a caching feature. And so it asks like which of those scripts are cacheable. So def and start probably not because that's really just about the development. But build definitely is. Potentially if we would have test or linting scripts as well, those would be too. So let's select build here. We could even customize the output folder if there is any, but most commonly this build, publish, public are already covered. And then also if we want to enable distributed caching. So we can definitely opt in here and install that. So now what we got here in this setup is we got a new nx.json. So first of all, there's a new dependency with nx here in our package.json at the very root. We have also this nx.cloud package because we opted in into the distributed caching. And the more interesting part here is that nx.json file, which has the cacheable operations defined and the target default as well. So this is exactly the type of operation which I mentioned before, which makes sure that whenever we run the build of some package or application, it runs the build of all its children first and then the build itself. So let's try this out immediately. If we, for instance, remove here auth and design system, the dist folders, and then we run the build of my next app. You can now also use the nx commands, which are more handy because we can just run nx build, and then here choose my org, my next app. And so you can see now it runs two dependent project builds, which are exactly our dependencies that we have. And then it runs the actual application build of our next app. And the cool part is like if we rerun this now, it is cached. So it would immediately be split out. And we can even configure this setup. So we can go ahead and say, we don't just want to run this for actual build command, but also if we run in development mode for our setup. So now that we have this workspace, this monorepo setup configured, how can we now leverage this migration part? Well, let's add a Remix app. So all I'm going to do is just cd into that apps folder and then run the create Remix app. So I call it my Remix app. I just want to have the basics Remix app server. I want to have TypeScript. And then I can just add it to my workspace. If I here navigate out to the root of my package and run npm again, it would set up and recognize now also my Remix application, which lives alongside my Next app. And so now that we have this set up, we can actually go ahead and just link the packages just in the same way we did in our Next.js app. So I can go ahead and here take these two packages, go to the Remix application, add those as well here into my system. And now I could go and import them from the Remix routes. So now we have our buttons here imported. We can just go ahead and say here, welcome to Remix. And add here, let's say our button just below it. Let's install the dependencies, just like npm can link everything together. And now we can run our Remix application. It will first again, pre-build our two dependent projects like the auth and design system library. Now if we go to localhost 3000, we have our Remix app running and we also have the button included. So what we have seen here is a very flexible setup where we can configure the tooling however we want. But in order to set, we also have to configure them. So this is what we call a package-based monorepo setup where all of these packages are potentially independent. They have their own dependencies. You can choose the build tooling that you want to have and you need to configure everything so that it works as it is intended to work. Now let me show you a different setup which we call integrated monorepo workspace, which NX is capable of setting up for you. And the difference there is it comes pre-configured, also more opinionated, but it has a set of plugins that help you in the development of your project. So in this case, again, we have our Next.js application, but for instance, also for our packages or libraries, we can generate them. We can preset them up and we don't have to worry about how the setup is being done. So by having these NX console extension installed, I actually can even go ahead and just right-click, select here a React library, which is my design system that I want to configure and have that generated for me. And so this would set up the design system in a certain way for me already with a lib folder, a couple of facilities in here that I can then consume from within my application. Similarly, I can go ahead and configure my auth library. So in this case, I could say this is really just a normal JavaScript library. There's no React in there. So let me choose here the novel.js plugin and define my auth package and just generate that as well. So all of these are automatically here configured and linked with a global root level TS config base. So rather than using here something like npm workspaces or pnpm or yarn workspaces, this is done over some TypeScript path mappings. The experience is actually exactly the same because we can still go to our Next.js app into the pages folder here, into our file, and just import something from that auth folder. So we can go here to auth and then whatever we have exported there, we can import it into my application. The experience here is very nice because what we can do here, for instance, if I return the name, I can go here and I can reference that. And so when I run my application, it automatically would refresh the application whenever I change something in that auth folder. So I can go ahead and just surf here my Next app. And so if I navigate to 4200, you see here my Next app being run. I see here John Doe. And if I go back to my auth application on my auth library, and add a couple of exclamation marks here, go back, you see it is automatically refreshed. So that is already wired up for you. You don't have to set up any pre-built steps or any watcher scripts, but this works out of the box. So what if I want to add now a Remix app to this? Well, I can just go ahead and install the Remix package, the Remix plugin here. And so once I have the plugin installed, I can go ahead and generate a new Remix application. So I can again run nx generate application. Now I should see the now Remix app. I give it the name, can give it a couple of options if they are available, and then I generate the Remix app inside my app folder as we did before. And so now that I have my Remix app set up here, I can just go ahead and actually also reference and import things the same way as I did in my Next.js application. So I go back to my Remix app inside the routes folder. I import auth package and just use it the very same way. Let's actually run the Remix app. So then if we load the Remix app, we can see it loads up, it shows John Doe, and it also works with live refresh. So if I go back to my auth part here, I remove the exclamation marks again. I go back, you can see it already refreshed and visualized the result for me. And so this is really just the tip of the iceberg here. The main advantage of such an integrated setup is that it pre-configures the stack for you. So you can actually focus on structuring your workspace, but you don't have to worry about how to build those, where to pre-build those. And moreover, it comes with facilities such as those generators, which we have seen, which can do even more such as here generating Remix actions, loaders, metadata for your routes and much more. So what's more? This is really a different approach to how you organize and structure your code in such a monorepo workspace. Because what happens here is that your application at the very top become very, very thin, and they're really just the bundling part of your tag specific setup. So this is where you include the components into your routing, how that specific tag, like the Next.js or Remix setup demands for it. While the packages down there are more encapsulated, they are well-defined, they have well-defined API boundaries usually, and they can even be tag agnostic. So they don't have to, because some of those might be feature specific things where we even expose like Remix loaders, right? Which you include in a Remix application. But things like the authentication workflows can totally be tag agnostic and be written in pure TypeScript. And so as you can imagine, this allows for much easier portability of features. And even though if you have to refactor things, it's much easier if they have already been extracted. What you also get most interestingly is that you can run, for instance, the visualization of a graph. This is specific to NX, where you can run an X graph and it would show you now the structure of your application. Now, in our super simple setup, we really just have two apps, which depend on both libs. But in a more evolved setup, this might get more complicated. And most importantly, also, this type of setup scales with your team. It is very easy now to, for instance, allocate teams to those specific packages where one works on the authentication part. There's another set of people and developers that work on the design system part that are more shared across different apps that you might have. But then there is also like features such as a product list where another team works on that is just focusing on the business aspect of the application. And also, now that you have those more fine grained packages, you can even run the tests in an isolated mode against just the product list, just the authentication part. So it's even easier to figure out what is going wrong if tests fail, because they are very easily identifiable. And moreover, if you now run this even on CI, given that graph that is now present, it is impossible to just run things that got touched. For instance, let's say we touch a single node in that graph, then only those would be actually tested and built and linted in your CI system. At the same time, this is also a way for diversifying your stack, because as we have seen, we start from an XJS application and have the intention of adding a remix app and then migrating pieces over. But you might even have multiple applications, multiple different tech stacks at the same time, which you deploy to different type of destinations of your organization. And this is all thanks to that modular structure where the apps are decoupled from the actual modular packages. So we particularly have also seen today how we can implement such a stack with an X and how that can help. Now we have seen two different approaches of how to use an X in this setup. The first one is probably more ideal if you have an existing stack already. So let's say you have that NPM workspace already in such a configuration, but now you want to add remix to it and you want to have an X to help you manage that and scale that as it grows. If you start new, you probably lean towards the more integrated type of configuration, which gives a lot of more benefits, such as all that fast refreshing, the watching already works. But at the same time, it gives you that pre-configured setup where you don't have to worry about all the lower level details on your own. The cool part here is really that an X helps you not only scale as you grow with features such as local and remote caching, but also comes with some nice DX aspects, such as like dedicated visual code extension that can help you develop it in the long run. So if this sounds interesting, definitely join us and follow us on NxDevTools, on Nx.dev, which is the main website, or also chime in into our community Slack, which you can find on the slide here. Thanks for watching.