Micro-scopes – How to Build a Modular Modern App in a Bundled World

Hi everyone, welcome to my talk, microscopes, about bundling code in the modern JavaScript world. But before we start the talk, a little bit, a small story. So this amazing image that you see right now, the Pillar of Creation, was being taken by a space telescope. And this space telescope is called Hubble. And Hubble was launched in 1990, costing 4.7 billion dollars to launch. And it was a huge project, maybe one of the biggest ones that humanity ever undertook. And it was launched, and it was amazing, and it didn't work, because something was broken. The camera was broken. So they couldn't fix it. They couldn't pull it back to Earth and fix it. So they had to train astronauts for three years to go up there and fix the camera. And today, we have a much simpler solution. So today, we're using nano satellites, nano satellites, hundreds of thousands of satellites that are going around the Earth. And they're really small. They're really cheap. It's really easy to launch each one of them. And if one of them gets broken, then we just replace it. We just decommission it, and we just replace it. And that's the genuineness of a modular architecture. Instead of sending one big thing, a monolith if you want, you send a lot of small things. And if something breaks down, you just fix them.

I'm Riyad Youssef. I'm the client architect at Dooda. And today, we'll talk a little bit about bundled world above and beyond. So how do we manage, how do we handle bundled JavaScript code? The first question you probably ask yourself is why? Why do we need to bundle at all? I have my code, I have it in the files, why do I need to bundle? And for that, we have to go a little bit back in history, a little bit back to maybe the dinosaur's age, where our code bases were unmaintained. And our files looked like that. We had huge files, a lot of lines. And if we did want to break into small files, we had to put a lot of script tags inside a document, and it was really tough to maintain. It was really tough to understand what's happening there.

So today, we want to write some sort of modular code. We have modules in the browser, but it's not fully ready yet, so we just write modules in our source code, right? We have a code, a file, let's say app.jsx, and it has dependencies. So it has app-reference dependency, top, bottom, and button. Maybe they have interdependencies between themselves, and maybe those dependencies have their own dependencies, and they connect between themselves. So how do we handle that? How do we handle all of these dependency chain, all of this dependency tree? We have Webpack. Webpack is today the standard way, and with an asterisk, because there are a lot of similar tools that make it a little bit better, but Webpack is the de facto standard for bundling modules. So we're talking a little bit about Webpack today, and we're talking a bit about how to use it. So just a little bit, a primer on Webpack, Webpack creates a single file, a single bundle, containing the entire code, and it does it, if we don't tell it, we just tell it, okay, we point it to the main entry, to the index, to the main file, and we tell it where to output.

Webpack works in this way. It goes to the entry, it tries to understand from the entry what are the dependencies, and it parses them, and recursively it builds the entire dependency tree and maybe Lodash has its own dependency, and then it just collects all of the dependencies, understands the dependencies, and bundles them. Bundles them in a nice package Bundle.js. It basically serializes the dependency with a little bit of magic from the top, so it serializes them to a single file, Bundle.js, and we can then take this Bundle.js file and use it in our HTML. We can put it in a script tag, because that's something that browsers know how to handle. But we have a problem, because Webpack, unless called otherwise, it will bundle everything to one file. So you can see here a dependency graph. It's an actual dependency graph of a medium-size application, and if you bundle all of this together, you get 15 megabytes of Bundle. So that's enough to get everyone desperate. But in Webpack, we can do code splits. So we can define for Webpack. We have split points in the code, so we don't want to bundle everything into one file, but we want, from these points, to bundle to this file, and then have a dynamic chunk. So split my code and include it only when I need it. So this is the easiest way to do that.

You just define, if you have sync import or sync required, it would be the same chunk. But if you have what you call async import or dynamic import, it will be a different chunk. And this is like a real-world sample of it. If you want to break into different parts, to break into different views, that's how you do it. That's how it looks under the hood if you do code splitting in Webpack.

So every color here represents a chunk, a chunk, or a bundle, or a part of a bundle. And you can see all the models that went inside this chunk. Having said that, we can use that in React, because we have lazy in React, and we can use those dynamic chunks. For example, if you want to bring the math component, only if Elon Musk is true, so we can use it with the lazy, and if we wrap it in a suspense, we get even a nice spin out while the code is trying to bring mass. And if we put earth inside the suspense as well, we'll have a spinner for everything. So lazy is a nice way, but how do we support more than one screen with this architecture?

Let's say we have three screens for application. We have the dashboard, we have the editor, we have the website. We just import them asynchronously, and we use React lazy, and in our application, we do a switch or an if, whatever you want. And then we just tell Waypack, hey, don't bundle everything into one file, but create dynamic chunks. So it's nice. That's what Waypack knows to do. It takes all the dependencies and it wraps even the dependencies, even the common dependencies it knows to wrap in different chunks.

And the benefits of it is that you have single entry point. You don't have to mess around with orchestration. You don't have code duplication, because Waypack is smart, so it doesn't duplicate code, but you have little influence on chunking or on the order of chunking. You can't partially build. If you want to build only the dashboard, you can't do it. You have to run the entire build for everything because you have one single point, single entry point, and it's the same repo, same language.

So let's talk about multiple entry points. So this is the example of a single entry point. But what if we do something like this. In Fig, we say, okay, don't bundle 100 points, but 3, editor, dashboard, and website. And then it will create 3 outputs, editor.js, website.js, and dashboard.js. It's still sharing the chunks. So these entry points, dashboard, editor, and website, and their inputs, eventually recreate those bundles, those outputs. But now we need to orchestrate, because now we have 3 different output files. It's on us to put them in the appropriate HTML. So we need to put them in the HTML. But why stop there? We can do multiple WebEx instead of doing multiple entry points. So multiple WebEx basically says, we do share the build, but we build every time a different entry points. So we have 3 build commands. It's not the same WebEx file. It's a different WebEx file. Every WebEx file has its own entry. Now we can control what goes into every file, every disk.

Now we can control what goes into every file, every disk. We can control the dependencies. We can even, what's called, dynamically build it. So for example, in this example, I'm taking the npm run build website. We'll only build the website.js and then npm run build dashboard. We only build the dashboard.js.

As you can see, the dependencies here are duplicated. And it's easy to see. Because WebEx doesn't know. When you run it like that, WebEx doesn't know that the React dependencies are shared. Because you run it separately. So you get full isolation. So dashboard and command will build dashboard.js. And that's fully isolated. You can use dashboard.js without even worrying about the website. And you have full control on the build. Because you can run. If something changes in the dashboard, you can build only the dashboard. That's OK. It's the same repo and package. As a developer experience perspective, it's really easy to maintain. And you can do incremental deploys. So again, building only what changes. The downside is that you duplicate the code. Because you duplicate the dependency. And you need orchestration. Again, because eventually, you don't have one entry point. You have multiple outputs. So you need to decide what output goes where. So one idea for orchestration, and I think it's a good one, is to output everything as a UMD.

UMD is a universal model definition. Meaning that you can consume it dynamically. You can consume it asynchronously. And then have a small models manager file. And its only purpose is to get the name of the model. And then do a require, like an AMD require, and returns a promise with this model.

And now it's super easy. Because now, if I'm a file that wants to use the editor, I don't care where the editor sits. I just go to the models manager. And I say, I need to get editor. And then the models manager will return the editor API, or whatever the editor exposes. And I can do with it whatever I want. And the dashboard the same. I'm just going to the models manager. And the models manager has all this knowledge. And you can even, in this architecture, you can even put it in S3. You can put it in the cloud. It doesn't matter, as long as the models manager knows what will define those files, it's enough. And that's really good. That's a really good idea. But it does duplicate the code.

And let's see an example here. So for example, we have all those characters. And we know that the Avengers are Captain America, Iron Man, Thor, Wanda, and Vision. But we know the Guardian of the Galaxy also needs Thor, right, because he's in the movie. And WandaVision needs WandaVision. So the thing is, if we have this dependency graph and we build it with these tactics of multiple WebPacks, we'll have duplicate code. Because the Avengers bundle will have Thor, Wanda, and Vision. But also, the Guardians of the Galaxy will have Thor, and WandaVision will have Wanda and Vision. And then if we try to put all of them together, it's just duplicated code.

So one useful tool in order to avoid duplication is to use Webpack externals. And externals basically tells Webpack, I have this dependency, but don't bundle it yourself. So don't try to bundle it. But I will provide it. So for example, here we say the externals that react and react-dom and Thor in this example. It means when Webpack sees the require of react or of Thor, it won't try to put it in the bundle. But it would rely on me to provide it for it.

So in this case, Thor is only one. There's only one instance of it. And I need to provide it. But again, it's on me to orchestrate it. I need to put the external script before I reuse my application. And I need to enforce separation of code. I need to enforce separation of dependencies. So this is how we do it.

We have an ESLint plugin. When you try to consume code from a different model, for example, if you're in the dashboard and try to consume something from the editor, we have an isWarning and the build fails because you've tried to import external model. We can see this is, for example, the doEditor. So this is the editor. This is the dashboard. And this is the model that we call the image speaker. It's the image speaker. And the reason that it's a different model, separate model, because it needs to be consumed from the dashboard and from the editor. So it's built independently.

So this is the editor. So it's easy to consume it. You just say, OK, we open an empty pop-up. Then we ask for the image speaker API or the image speaker whatever it exposes. And then I just render it into the pop-up container. And that's it.

And our entire application consists of different models, multiple WebEx. So it's a really good way. It's a really good approach if you have a medium sized app and you still want to keep your code in the same repo. But you do want to separate it. You do want to build it separately. You want to deploy it separately. So you have to have these kind of mechanisms in order to avoid cross model imports, like we saw, like the ASLint plugin. But once you have it in place and you have the external mechanism in place, it's really easy to maintain. Because the team that is working on dashboard is working only on the dashboard part. And it can consume common if they want. But it doesn't interfere. And you can see here how easy it is to consume.

If it sounded familiar, then it's because it's not very far from something we call a micro-fontans. And micro-fontans is going a little bit further than what I just described. So in my example of multiple WebEx, we have one repo, one repository, where all the code lies. And we just build different parts of it separately.

Micro-fontans says we want to assemble the application from independent parts. So for example, if we speak about the ISS, International Space Station, it's a huge thing for us to have in space. So we didn't just send it, we assembled it from independent parts, every country built it independently, launched it independently, et cetera, et cetera. So micro-fontans will allow us to even use different languages or different franchises in our app. Every team can write in JavaScript or Go or whatever they want, because micro-fontans doesn't care about the code itself. For example, this kind of website, this kind of web app, we can just break it, we can just split it into different parts. And we can have different teams, or even in different places build those parts. And they can be in different repos, even in different technologies, different frameworks. And we don't care, all we need is the API to call the micro-fontan.

But here is where it gets tricky. And this is how it looks like when you assemble them. So you have different repos, so you can obviously do separate builds and deploy because they don't know about each other. You have full autonomy for every repo, for every team, they can decide how they want to do this process. Orchestration is super important. And you have to do some sort of dependency management. Why? Because it's different repos. So you can't share common code, like we said in the multiple webpack entries. So you have to have common code, either as a package, as a separate package, like your component library, or as an external if it's a library that you want to consume as an external. You have to have orchestration because you need to know versioning, you need to know the version of every microfrontend. You have to have discoverability because every microfrontend needs to know where are the other ones, and composition. You have frameworks to do that, or you have libraries to do that. Just to mention a few, you can write it one yourself or use single SPA, or use the idea of iFrames. So every microfrontend goes into iFrames or have server side includes the options are limitless.

So you would say, okay, we have the problem solved but it's not that easy, right? Because who wants to maintain hundreds of packages or even dozens of packages, each in different people, small packages that you need to download the code and then write it and then commit, and then do a PR and then wait for the publish. You don't really want to do that, no one wants to do that. So we have one or repo for the rescue because many of the truths that we cling on depend on our point of view, and we can just put it back in the same repo.

So we have different packages, but same repo. Again, we have the versioning problem. How do we know what the version of each one? But again, we have tools like Learner. It's a very powerful tool that allows you to build all of them together, to manage versioning together, to deploy, to test everything together. Learner is, I super recommend to check it out if your app is one that you need to use Monorepo, it makes the working with Monorepos much easier. There's also buildvilt, that's also checking.

So now we talk about separation versus the duplication. Right, we need to decide, it's like a trade-off. We need to decide, do we want to separate the code, or can we risk in duplicating the code? Because let's do a small recap of what we saw until now. You can either tell webpack, okay, bundle everything and just take care of the dependencies yourself and just take care of the chunking yourself. And that's awesome and that's really good. That's what you get when you run, for example, Create React App, that's what you get out of the box because you have one single entry point, you don't need to worry about it. But then when you start building a bigger application, when you start saying, okay, maybe I have different teams or I have different people working on different areas, you do want the ability to decide what you want to build or what you want to deploy if you have different parts of the applications. And in a single entry point, you can't do that. So you can do multiple entry points like we saw, you can tell Webpack, hey, build those entry points or just build the entry point that I want. And then you have a little bit more freedom, but then you have the headache of orchestrating it, of saying, okay, I built my dashboard, I built the editor, I built the image picker, but now I need to write the code that knows where everything is and then consume it. And also you lose all the dependency management or the common dependencies management that Webpack gives.

So one solution is model federation, which is really cool. Model federation is something new that Webpack releases. And let's say that app one is a host app and it wants to consume something from app two, but it doesn't want to duplicate it. App one is the shell, app two is the orange one and it want to consume the button. So app one just defines in its config, it says, hey, I am app one, I'm using a model federation, it's a new thing in Webpack and my remote is app two. So I want to consume button from app two, but I don't want it to be bundled in my code. Okay, and I'm willing to share React and React DOM with it. App two is the remote says I'm app two, I'm exposing the button and I'm also I can share React and React DOM. And then in app one, we just import it from app two, app two slash button. And in app two, we just import it locally. And it's really amazing because in this sort of mechanism, we can do even something more complex. We can have a lead app that only exposes React and React DOM the libraries. We can have component app that only exposes components and we can have apps that uses those components.

And instead of bundling everything together, we just define the appropriate Webpack config files. So we define them appropriately for example, main app is consuming libapp and component app. And then we just consume them and we have this dependency fixed. It's like magic.

So you can read more about it everywhere around the web. There are a lot of other techniques, for example, web packaging or web bundles. It's like an unstable thing that allows you to bundle a website into one file. And the important thing is that to remember that you don't need to decide between code separation and duplicate dependencies, you can have both and the future is really amazing. So just look ahead and thank you very much.

I'm Liad Yussef. You can find me on the Twitter if you want. Thank you very much.