Micro-scopes – How to Build a Modular Modern App in a Bundled World


In this talk we will explore the great benefits of breaking a big modern app to meaningful, independent pieces – each can be built, deployed and loaded separately. We will discuss best practices and gotchas when trying to apply this microservice-like pattern to the chaotic world of the browser, and we'll see how building the right pieces guarantees a brighter future for your apps. Let's dive into Neverendering story of modern front-end architecture.


Hi everyone. Welcome to my talk, Microscopes, about bundling code in the modern JavaScript world. But before we start the talk, a little bit, a small story. So this amazing image that you see right now, the Pillars of Creation, was being taken by a space telescope. And this space telescope is called Hubble. And Hubble was launched in 1990, costing $4.7 billion to launch. And it was a huge project, maybe one of the biggest ones that humanity ever undertook. And it was launched, and it was amazing. And it didn't work, because something was broken. The camera was broken. So they couldn't fix it. They couldn't pull it back to Earth and fix it. So they had to train astronauts for three years to go up there and fix the camera. Today we have a much simpler solution. So today we're using nanosatellites. Nanosatellites, hundreds of thousands of satellites that are going around the Earth. And they're really small, they're really cheap. It's really easy to launch each one of them. And if one of them gets broken, then we just replace it. We just decommission it, and we just replace it. And that's the genuinity of modular architecture. Instead of sending one big thing, a monolith if you want, you send a lot of small things. And if something breaks down, you just fix them. I'm Liad Yosef. I'm the client architect at Duda. And today we'll talk a little bit about bundled world above and beyond. So how do we manage, how do we handle bundled JavaScript code? The first question you probably ask yourself is why? Why do we need to bundle at all? I have my code, I have it in the files, why do I need to bundle? For that, we have to go a little bit back in history. A little bit back to maybe the dinosaur's age, where our code bases were unmaintained and our files looked like that. We had huge files, a lot of lines. And if we did want to break into small files, we had to put a lot of script tags inside the document. And it was really tough to maintain. It was really tough to understand what's happening there. So today we want to write some sort of modular code. We have modules in the browser, but it's not fully ready yet. So we just write modules in our source code. We have a code, a file, let's say app.js6, and it has dependencies. So it has app.reference.dependency, top, bottom, and button. Maybe they have interdependencies between themselves. And maybe those dependencies have their own dependencies, and they connect between themselves. So how do we handle that? How do we handle all of this dependency chain, all of this dependency tree? We have Webpack. Webpack is today the standard way, and with an asterisk, because there are a lot of similar tools that make it a little bit better. But Webpack is the de facto standard for bundling modules. And we'll talk a little bit about Webpack today, and we'll talk a little bit about how to use it. So just a little bit of a primer on Webpack. Webpack creates a single file, a single bundle, containing the entire code. And it does it if we don't tell it how. We just tell it, OK, we point it to the main entry, to the index, to the main file, and we tell it where to output. And Webpack works in this way. So it goes to the entry. It tries to understand from the entry what are the dependencies. And it parses them, and recursively, it builds the entire dependency tree, and maybe Lodash has its own dependency. And then it just collects all of the dependencies, understands the dependencies, and bundles them in a nice package, bundle.js. It basically serializes the dependency with a little bit of magic from the top. So it serializes them to a single file, bundle.js. And we can then take this bundle.js file and use it in our HTML. We can put it in a script tag, because that's something that browsers know how to handle. But we have a problem, because Webpack, unless called otherwise, it will bundle everything to one file. So you can see here a dependency graph. It's an actual dependency graph of a medium-sized application. And if you bundle all this together, you get 15 megabytes of bundle. So that's enough to get everyone desperate. But in Webpack, we can do code splits. So we can define for Webpack, we have split points in the code. So we don't want to bundle everything into one file. But we want from this point to bundle to this file, and then have a dynamic chunk. So split my code and include it only when I need it. So this is the easiest way to do that. You just define, if you have sync import or sync require, it will be in the same chunk. But if you have what you call async import or dynamic import, it will be a different chunk. And this is like a real-world sample of it. So if you want to break into different parts, to break into different views, that's how you do it. That's how it looks under the hood. If you do code splitting in Webpack, so every color here represents a chunk, a chunk or a bundle or a part of a bundle. And you can see all the models that went inside this chunk. Having said that, we can use that in React, right? Because we have lazy in React. And we can use those dynamic chunks. For example, if you want to bring the mask component only if Elon Musk is true, so we can use it with the lazy, with lazy. And if we wrap it in a suspense, we get even a nice spinner while the code is trying to bring Mars. And if we put Earth inside the suspense as well, we'll have a spinner for everything. So lazy is a nice way. But how do we support more than one screen with this architecture? So it's easy. Let's say we have three screens for application. We have the dashboard, we have the editor, we have the website. We just import them asynchronously and we use React lazy. And in our application, we do a switch or an if, whatever you want. And then we just tell Webpack, hey, don't bundle everything into one file, but create dynamic chunks. So it's nice. That's what Webpack knows to do. It takes all the dependencies and it wraps even the dependencies, even the common dependencies it knows to wrap in different chunks. And the benefits of it is that you have a single entry point. You don't have to mess around with orchestration. You don't have code duplication because Webpack is smart. So it doesn't duplicate code. But you have little influence on chunking or on the order of chunking. You can't partially build. If you want to build only the dashboard, you can't do it. You have to run the entire build for everything because you have one single point and single entry point. And it's the same repo, same language. So let's talk about multiple entry points. So this is the example of a single entry point. But what if we do something like this in Fig? We say, OK, don't bundle 100 points, but three, editor, dashboard, and website. And then it will create three outputs, editor.js, website.js, and dashboard.js. It's still sharing the chunks. So these entry points, dashboard, editor, and website, and the reports, eventually recreate those bundles, those outputs. But now we need to orchestrate because now we have three different output files. So it's on us to put them in the appropriate HTML. So we need to put them in the HTML. But why stop there? We can do multiple Webpacks instead of doing multiple entry points. So multiple Webpacks basically says we do share the build, but we build every time a different entry point. So we have three build commands. It's not the same Webpack file. It's a different Webpack file. Every Webpack file has its own entry. And now we can control what goes into every file, every disk. We can control the dependencies. We can even, what's called, dynamically build it. So for example, in this example, I'm taking the npm run build website. We'll only build the website.js and npm run build dashboard. We only build the dashboard.js. As you can see, the dependencies here are duplicated. And it's easy to see because Webpack doesn't know. When you run it like that, Webpack doesn't know that the React dependencies share it because you run it separately. So you get full isolation. So dashboard and command will build dashboard.js. And that's fully isolated. You can use dashboard.js without even worrying about the website. And you have full control on the build because you can run if something changes in the dashboard, you can build only the dashboard, and that's okay. It's the same repo and package, so it's really easy to, as a developer experience perspective, it's really easy to maintain. And you can do incremental deploys. So again, building only what changes. The downside is that you duplicate the code, right, because you duplicate the dependencies. And you need orchestration, again, because eventually you don't have one entry point, you have multiple outputs. So you need to decide what output goes where. So one idea for orchestration, and I think it's a good one, is to output everything as a UMD. UMD is a universal model definition, meaning that you can consume it dynamically, you can consume it asynchronously, and then have a small models manager file. And its only purpose is to get the name of the model, and then do a require, like an AMD require, and returns a promise with this model. And now it's super easy, because now if I'm a file that wants to use the editor, I don't care where the editor sits. I just go to the models manager, and I say, I need to get editor. And then the models manager will return the editor API, or whatever the editor exposes, and I can do with it whatever I want. And the dashboard the same. I'm just going to the models manager, and the models manager has all this knowledge. And you can even, in this architecture, you can even put it in S3, you can put it in the cloud. It doesn't matter, as long as the models manager knows where to find those files, it's enough. And that's really good, that's a really good idea, but it does duplicate the code. And let's see an example here. So for example, we have all those characters, and we know that the Avengers are Captain America, Iron Man, Thor, Wanda, and Vision. But we know that Guardian of the Galaxy also needs Thor, right? Because he's in their movie. And WandaVision needs Wanda and Vision. So the thing is, if we have this dependency graph, and we build it with these tactics of multiple Webpacks, we'll have duplicate code. Because the Avengers bundle will have Thor, Wanda, and Vision, but also the Guardians of the Galaxy will have Thor, and WandaVision will have Wanda and Vision. And then if we try to put all of them together, it's just duplicated code. So one useful tool in order to avoid duplication is to use Webpack externals. And externals basically tells Webpack, I have this dependency, but don't bundle it yourself. So don't try to bundle it, but I will provide it. So for example, here we say the externals are React and React DOM and Thor in this example. It means when Webpack sees the require of React or of Thor, it won't try to put it in the bundle, but it would rely on me to provide it for it. So in this case, Thor is only one. There's only one instance of it, and I need to provide it. But again, it's on me to orchestrate it. I need to put the external script before I use my application. And I need to enforce separation of code. I need to enforce separation of dependencies. So this is how we do it. We have an ESLint plugin. When you try to consume code from a different model, for example, if you're in the dashboard and try to consume something from the editor, we have this warning and the build fails because you tried to import external model. We can see this is, for example, the Duda editor. So this is the editor. This is the dashboard. And this is the model that we call the image picker. It's the image picker. And the reason that it's a different model, a separate model, because it needs to be consumed from the dashboard and from the editor. So it's built independently. So this is the editor. So it's easy to consume it. You just say, OK, we open an empty pop-up. Then we ask for the image picker API or the image picker whatever it exposes. And then I just render it into the pop-up container. And that's it. And then our entire application consists of different models, multiple WebEx. So it's a really good way. It's a really good approach if you have a medium-sized app and you still want to keep your code in the same repo. But you do want to separate it. You do want to build it separately. You want to deploy it separately. So you have to have these kind of mechanisms in order to avoid cross-modal imports, like we saw, like the ASLint plugin. But once you have it in place and you have the external mechanism in place, it's really easy to maintain. Because the team that is working on the dashboard is working only on the dashboard part. And it can consume common if they want. But it doesn't interfere. And you can see here how easy it is to consume. OK, if it sounded familiar, then it's because it's not very far from something we call microfontains. And microfontains is going a little bit further than what I just described. So in my example of multiple WebEx, we have one repo, one repository, where all the code lies. And we just build different parts of it separately. Microfontains says we want to assemble the application from independent parts. So for example, if we speak about the ISS, the International Space Station, it's a huge thing in space. So we didn't just send it. We assemble it. We assemble it from independent parts. Every country build it independently, launch it independently, et cetera, et cetera. So microfontains will allow us to even use different languages or different franchises in our app. Every team can write in JavaScript or Go or whatever they want. Because microfontains doesn't care about the code itself. So for example, this kind of website, this kind of WebApp, we can just break it. We can just split it into different parts. And we can have different teams or even in different places build those parts. And they can be in different repos, even in different technologies, different frameworks. And we don't care. All we need is the API to call the microfontain. But here is where it gets tricky. And this is how it looks like when you assemble them. So you have different repos. So you can obviously do separate builds and deploy because they don't know about each other. You have full autonomy for every repo, for every team. They can decide how they want to do this process. Orchestration is super important. And you have to do some sort of dependency management. Why? Because it's different repos. So you can't share common code, like we said in the multiple WebPack entries. So you have to have common code either as a package, as a separate package, like your component library, or as an external if it's a library that you want to consume as an external. You have to have orchestration because you need to know versioning. You need to know the version of every microfontain. You have to have discoverability because every microfontain needs to know where are the other ones and composition. You have frameworks to do that, or you have libraries to do that. Just to mention a few, you can write one yourself, or use single spot, or use the idea of iframes. So every microfontain goes into iframes, or have server-side includes. The options are limitless. So you would say, OK, we have the problem solved. But it's not that easy, right? Because who wants to maintain hundreds of packages, or even dozens of packages, each in different repos, small packages that you need to download the code, and then write it, and then commit, and then do a PR, and then wait for the publish? You don't really want to do that. No one wants to do that. So we have one repo for the rescue because many of the tools that we cling on depend on our point of view. And we can just put it back in the same repo. So we have different packages, but same repo. Again, we have the versioning problem. How do we know what the version of each one? But again, we have tools like Lerna. It's a very powerful tool that allows you to build all of them together, to manage versioning together, to deploy, to test everything together. Lerna is, I super recommend to check it out. If your app is one that you need to use Monorepo, it makes the working with Monorepos much easier. There's also build, B-I-L-T. That's what's checking. So now we talk about separation versus duplication, right? We need to decide. It's like a trade-off. We need to decide, do we want to separate the code, or can we risk duplicating the code? Because let's do a small recap of what we saw until now. You can either tell Webpack, okay, bundle everything and just take care of the dependencies yourself and just take care of the chunking yourself. And that's awesome. And that's really good. And that's what you get when you run, for example, create React app. That's what you get out of the box because you have one single entry point. You don't need to worry about it. But then when you start building a bigger application, when you start saying, okay, maybe I have different teams or I have different people working on different areas, you do want the ability to decide what you want to build or what you want to deploy if you have different parts of the applications. And in a single entry point, you can't do that. So you can do multiple entry points like we saw. You can tell Webpack, hey, build those entry points or just build the entry point that I want. And then you have a little bit more freedom, but then you have the headache of orchestrating it, of saying, okay, I built my dashboard, I built the editor, I built the image picker, but now I need to write the code that knows where everything is and consume it. And also you lose all the dependency management or the common dependencies management that Webpack gives. So one solution is model federation, which is really cool. Model federation is something new that Webpack releases. And let's say that app one is a host app and it wants to consume something from app two, but it doesn't want to duplicate it. App one is the shell, app two is the orange one, and it wants to consume the button. So app one just defines in its config. It says, hey, I am app one. I'm using a model federation. Again, it's a new thing in Webpack. And my remote is app two. So I want to consume button from app two, but I don't want it to be bundled in my code. And I'm willing to share React and React DOM with it. App two is the remote says, I'm app two. I'm exposing the button, and I'm also I can share React and React DOM. And then in app one, we just import it from app two, app two slash button. And in app two, we just import it locally. And it's really amazing because in this sort of mechanism, we can do even something more complex. We can have a lib app that only exposes React and React DOM, the libraries. We can have component app that only exposes components. And we can have apps that uses those components. And instead of bundling everything together, we just define the appropriate Webpack config files. So we define them appropriately. For example, main app is consuming lib app and component app. And then we just consume them. And we have this dependency fixed. It's like magic. So you can read more about it everywhere around the web. There are a lot of other techniques. For example, web packaging or web bundles. It's like an unstable thing that allows you to bundle a website into one file. And the important thing is to remember that you don't need to decide between code separation and duplicate dependencies. You can have both. And the future is really amazing. So just look ahead. And thank you very much. I'm Liad Yossef. You can find me on Twitter if you want. Thank you very much.
21 min
11 Jun, 2021

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Workshops on related topic