Module Federation in Webpack 5

Thanks for the introduction. Hello, my name is Tobias Koppers and I work for the webpack Core team. And I'm going to talk about module filibration in webpack 5. So my talk is about module filibration in webpack 5. And I want to tell you about the motivation for this feature, how it works, and how to use it. So the motivation is you have a large to mid-scale application, and you work with multiple teams on a multiple application path. You have separated your application or applications into multiple paths, like micro-fonts, but also logical paths. And these paths should be developed independently by different teams. And another requirement is that you have multiple paths sharing common libraries or sharing on other paths. Here's an example. You could have a header component, a sidebar component, as micro-fonts or pages. Or you could also have a style guide components library, which is shared by all the applications. But also different things than font ends, like data fetching logic, business logic, or other logical components. So let's look at the existing options with webpack. You could just go with native ECMAScript modules. This way you don't have any build process for linking the path together. You could just consume natively all your modules in your applications, and it would use native browser import statements to link them together. But there are some challenges with this approach. You basically opt out of all optimizations webpack would do for you, like unused explores, concatenation of modules, other optimizations. And there are also some challenges about web performance. Like you would have each module separately, so it causes a high count of requests at runtime. Each request has an overhead, and you get less effective compression with more requests, with smaller files. Because larger files usually compress better than smaller files. And you have also this drawback about only being able to use ECMAScript modules, and you can't use CommonJS modules, you can't use css modules, ESSERTS, or other loader process things like other languages also in webpack. There's this option about just doing a single build of all your applications and all its parts together. This way every module of every other part or application is accessible during the build process, so you can just use them via import statements. But there's also a few challenges with that approach. Each update requires a full build of all applications and all parts, so it has a high build time, and this means high deploy delay from update to deploying the new version of the application. And you also have this problem that you can't separately build each application, so your applications don't stay separate from each other, because if you want to share common parts or common modules or common libraries at one time, you have to build them together. So it's a challenge you have to come up with. But there's a plugin in webpack which allows you to separate a part of your build process into a separate build, which can be built independently. In this scenario, you would build each part as DLL, with a so-called DLL plugin. And these DLLs can be consumed at one time by the consumer consuming build. But you also have a compile time dependency of the DLL-generated manifest at compile time, so that's also one challenge. You have to rebuild your application or part when a part has changed, all consumers have to be rebuilt. In this case, an additional deploy delay is not so high compared to the single build approach, but it's still an additional deploy delay you don't want to have. And there's also a big challenge about sharing libraries or sharing common modules. Basically, if multiple parts share a library, you have to pull out or extract this shared library into a separate DLL and separate build process manually, and then consume the DLL generated by the separate process by all parts sharing this library. So it's a lot of manual work involved to be able to share libraries between parts. An alternative to this is externals and built-in libraries. So in this scenario, each part would be built as a library and then consumed by the consuming parts of the application as externals. This eliminates this compile time dependency between parts and consuming parts, and other modules could be just consumed from the library at runtime. But still, the challenge about sharing libraries stays true for this scenario. Each shared library has to be extracted into a separate build process, separate library, and then be external in each of these consuming parts or consuming applications. So to summarize this, native ECMAScript modules are problematic because of web performance. A single build process is problematic about build performance, and the DLL and externals approach would work, but they require a lot of manual work to extract shared libraries or so on. So in the end, we need a scalable solution, or at least a trade-off, which has good build performance, good web performance, but also a good solution for shared dependencies. That's why we enter module federation. In module federation, you would build each part separately, and here we would build a so-called container, and each part would be published or deployed as a container. Any application or other containers could consume modules from this container. In this relationship, the consumer is the host and the container would be the remote. If the host consumes exposed modules from the container, then they would be called remote modules. So we got back again to each part is built separately, independently, and deployed independently. So we have this good build performance. You only pay for the build time of your part you're developing or you update it. And the whole process looks like this. Module federation comes with two features or aspects. The first aspect is exposing modules. A container can expose modules, and they can be consumed by the host via remote modules. An exposed module would be an asynchronously exposed. So this means if you request a module from the container, this would be an asynchronous process, and the container would then only load the code related to your exposed modules. This means you only have to pay the cost of the container and the cost for the modules you're really using from the container. So only the code is downloaded from the modules you are using. But as a container, you can still do optimizations like bundling dependencies of exposed modules together or extracting shared parts of exposed modules automatically. Also, optimization you can do for automatic loading is possible for exposed modules. So this brings back the good build performance by minimizing requests and also only loading what you're really using. So the second aspect is the shared modules aspect. In this aspect, each participant of the application landscape can provide modules, provide shared modules into the share scope. These modules are provided with version information attached. So the container could expose react in a version 60.3, and that would be put into the share scope. And on the other side, containers or consumers, every participant of the landscape can consume modules from the share scope with a version check. So the container may be asked for react in a version 60.0 or higher, and then it would get the highest version available in the share scope. This way, shared modules can be deduplicated within the share scope by version tagging, and you don't have to download react twice if you have a shared module. For shared modules, the same asynchronous loading is there like with exposed modules. You would put every shared module in a separate file or separately download it, and you only have to download shared modules you are really using. So you can provide older react versions, but if you're only using the newer version, it would not download any old version. So here's an example of this in work. So we start with just this normal application, which has a homepage, and on the homepage there's a login link, and the login link opens the login module and shows a button, which is called login also, and on the homepage there's a dropdown, which shows something. Everything uses react in this scenario, but the login module code is loaded asynchronously when clicking on the login button. So it's using on-demand loading for this scenario to move this into a separate chunk, move it and separately download it. In this scenario, we have two teams working on this. Team A is working on homepage and login module, so these components, and team B is working on the component library, which has a dropdown component and also the button component. This is how it works with a single build. Everything is super optimized, but let's apply module federation on this concept. So from the view of team B, team B only cares about their own components, like the button component and dropdown component, and also their dependencies, like an arrow icon is used for the dropdown and also the react as library. To use module federation, team B would flag these modules in the graph, like button is exposed, dropdowns are exposed, so exposed means they are available by consumption from the container interface, and also flag react as shared library, so they may be shared with other teams at one time. So now webpack would create a build, would build a container for all of this. So in this container, there would be a container entry module generated by webpack, which contain references to all the exposed modules, like button or a dropdown, and webpack would also put every exposed module into a separate file, so like here is button in a separate file, but also dropdown in a separate file, but still be able to bundle dependencies into the togetherness of the exposing module, so chunking optimization still applies. It would also put every shared module into a separate file, because it may be loaded or may not be loaded, depending on if already a react version is available at one time. So if you take some examples, like if you request the button component from the container, the runtime would load button chunks, so the file would contain the button, but also the shared modules of this required by this chunk, they would load them in parallel. In the scenario that if there is already a react version, same or higher, at one time available, then the request and dropdown would only require their own dropdown chunk, and the react chunk would be loaded from somewhere else, or not loaded at all if it's already been loaded before. So now let's team A use this container generated by team B. This is how the module graph looks for team A. Team A only cares about their own components, and each component from team B would be added as remote modules. So I'm only saying there's a container somewhere, and a consuming dropdown from the container. So from webpack point of view, there's a remote module, like the dropdown module, and every remote module points to the container as external at runtime. But there's a challenge here, because loading modules from the container is asynchronous, we have to be a bit creative to solve this problem, because if you log in module just imports the button component, and importing is usually a synchronous operation, so webpack will automatically hoist every asynchronous operation required for loading remote modules up to the next asynch boundary. An asynch boundary is like an asynchronous import statement or something like that. So in this case, if you click on the login link, which usually loads the code for the login module, it will in parallel load the code for the button component from the container. So you click the login link and log in module code, and button component code would load in parallel once from the local build, once from the container build. Yeah, and the same happens for shared modules, but it's not in detail explained here. Cool. So now, how can I use it? To use it, there's a module federation plugin available in webpack 5, and with different properties you have access to creating container, consuming other containers, but also sharing modules at runtime. Let's look at creating a container. So to create a container, you have to expose modules from the container, which is done by using the exposes property. So the exposes property has some properties like, give each module a public name, like tracking system or data, and a local name, which is where the module is on the disk currently from this build. And each module is supported, it could be just a normal ECMAScript module, it could be a common JS module, it could be css, it could be anything processed by loaders, what else. And to consume other containers, you have to use the remotes key, or remotes properties in the module federation plugin. And here you give each container a name like analytics, and also point to a container location, which is then loaded at runtime here, it would load the analytics JS script at runtime. To use these remote modules from containers, you would just have an import statement, which is privileged with analytics, and then the public name of the module in the container, of the expose module in the container here, tracking system. And to share modules, you have the shared property, and with this shared property you just list all the modules you want to be able to share between other containers or other applications. Like an example here, react Virtualize is shared, but also react Visualize, the styles of react Virtualize. And there's also advanced configuration available. Here an example is react, react must only be a law instance at once on an html page, so you can use a single advanced configuration to make sure react is only your loaded ones. For each shared module, webpack will figure out which version is provided by looking up the version information from the package JSON, but also would look up the required version for each dependency by looking in your package JSON in the dependencies list or the dependencies list. But you could also use more advanced configuration to override this or pass other things or define fallback modules, define different keys. There's also advanced configuration for container building and also for consuming containers available. If you're interested, pause the video and look into the details. At large scale, this could look like this. You have multiple applications which are using component libraries, using pages as separate containers, but also business logic is shared or translation stuff as well. There's also this funny aspect about orchestration. Now you have built all these containers separately, but you have to put them together at one time. There are basically two techniques I came up with. The first technique is evergreen. This means each application always uses the latest published version of a container. In this scenario, if you publish a change to your design systems components, then every application would instantly get the latest version of this at runtime, and there's no additional step between that. It's like having a package JSON with a log file and running npm install every time you load up the application in the browser. It's a bit risky, but it could work on a smaller scale because your company is in control of all your containers. If you don't make a mess, this could work for you. But there's also another approach I call it merged. In this scenario, applications would log the version of the container they are using and it's like having a package JSON with log file. And just like with a log file, there's an active step to upgrade or update the version of your containers which your application is using. You can validate if this update to your containers works for your application. This is where you can test on application level if containers are still compatible with your applications. And this could be multiple stages like staging, production, environment, and so on. Here are some ideas how to solve this, but I don't want to go into detail. You can pause the video if you're interested. To make a summary, module federation allows you to make different builds act as monolithic landscape at runtime. Any webpack supported module is supported like ECMAScript, CommonJS, but also ESSERTS, loader process stuff, or css. Sharing modules is available due to semantic versioning at runtime. It's available for all webpack types like Webnode, Webworker, etc. It will only load the modules you're really using, but it will never use all exports of them, so you have to be aware that there's a trade-off about optimization here. And federated modules are asynchronous, so you need to have an asynchronous loading boundary somewhere in the graph before you're able to use federated modules. So, our feature is experimental, so maybe break-and-trace is coming in the future, maybe there are more advanced features in the future. Here I already have some ideas of what could come, but I don't want to go into detail now. So, here are some of the sources. If you're interested, look up into the documentation, other talks, and more examples are available online. So, I have to say thanks. But let's go deeper into the topic of the federation. Tobias, can you join me on stage? Sure. Thanks for inviting me. Yeah, good to have you. I have to say, you're a brave man tackling these issues. So, I don't wear a hat right now, but hats off to you. And awesome that you made it understandable for a person like me who's not into these build systems. We're going to dive into our audience question. The first one I have is from Albert G. Is there any chance webpack will make it possible to integrate ES modules as DLLs or convert the webpack proprietary module system to be compatible with ES modules? So, I think with DLLs he means containers from the federation also. So, yes, there are plans to make webpack ECMAScript modules supported. Currently, we can use ECMAScript modules as external. So, if you have a container in the ECMAScript format, you would be able to launch this container. But we can't produce a container in ECMAScript format. Basically, the ECMAScript module format as output format for webpack is not enabled yet. We don't have it yet. We are working on this for webpack 5, and I think it will be available for the final release. But there are some technical challenges, not about basic JS that would be straightforward, but if you have combined it with css or this complex stuff. Like if you have interrupt logic in the model like common JS and questions about strict mode, like ECMAScript modules are always in strict mode. Common JS usually can opt into strict mode, so it would have some behavior changes to module. So, these are questions we are considering. So, it's kind of very difficult to make some feature like this. This comes with other bundlers like straightforward, but with webpack we have a lot of features we have to support if we want to provide something like that. So, it's not that easy for us to add this, but we are planning to add this and we probably add some limitations to that. And so, you can't use it with some complex structures, but we have some workarounds to make it work. It kind of ties into the panel discussion we had before. You don't want to be stuck in the past, you want to continue going forward, but you can't leave everyone behind. So, that of course slows you guys down, you people down as well. I think a lot of value in webpack is the stability and the backward compatibility for existing code. So, we have a lot of large user base and we don't want to leave them back with newer versions. We want to keep the existing features supported. Thank you. Thank you for that. We're going to go to the next question from Tudor. Are there any benchmarks on bundle size or load time results when using module federation? So, I don't have benchmarks, but Segjection has a big repo with a lot of examples and you can play around with a lot of sample cases with module federation. But to summarize it, we want to keep, so basically webpack is based on if you load something or load something on demand, it should only take one round trip to the server and download everything and maybe make a parallel request to load all the stuff. And we want to keep this limitation or keep this goal for module federation, so it will still take one round trip to the server. But with module federation, you could lose some optimization ability. So, it could increase the code size if you share modules with others. You have to provide all exports of the modules. We don't know which exports are used in the whole application landscape, so we basically prepare all modules to be used by anyone. So, we can do less optimization at this boundary between where shared modules or exposed modules come together with other one-time loaded containers or applications, which we don't know at compile time. It could increase bundle size, but it also could decrease it by having the ability to share modules between these micro-funded parts of the application. So, in the short answer, it depends. It depends. Yeah, that's always hard, of course. It depends on the landscape of the user. Always measure. Yeah, it's always the user. The next question is from Amalia. Is there any way to figure out the affected apps based on updating a container? Yeah, so, there's two different approaches in my talk. So, evergreen, so you don't get to know when a container changes. But the managed approach is about verifying that the application still works with updated containers. So, in this approach, you would actively test your application with the updated containers, make sure everything works, and then deploy the new application specification with the new latest version of the container. So, this is the way you want to do if you have considerations about stability when containers are updated and so on. Okay. We have a little bit more time. Let's just take another question. This question is from Kirill. Not sure if I understood. How does containers share between frontends? Are they deployed separately? Also, would loaded modules share state or there will be execution in isolation? They will be executed in isolation. So, regarding sharing state, so all the modules in all containers will act as monolithic applications. So, it will behave like if there was a single build of all the modules. So, all modules are basically kind of in the same cache. So, every module is only instantiated once, and they can share functions. They are not isolated. They are all put together in a common module scope or module thing. So, you can do everything you can do with normal state sharing. You can put state in your modules like a store, a Redux store, for example, and all applications, all containers will be able to access this state in this one module exposed or shared. What was the other part of the question? How are containers shared between frontends? Are they deployed separately? Yes, so the easiest way is to build every container separately and deploy it to some location, to some URL, and then you would use this URL to reference the container from applications or other containers. So, each container can be, and that was the intention of the whole system, deployed separately, and at runtime, they will be linked together and act as all applications. Okay, thank you. Last question we have time for is from Stefan. Can the bundles, modules be changed at runtime? For example, if I have 100 AP tests and want to ship only five to my users, which 51 only, which five I only know when the user requests my app? Can module federation help me with this? Yes, you can do this. You could compile two versions of a container and then choose at runtime which container you want to load. So, basically, you have your A and the B version, and at random or with cookies or whatever, at some condition, it wants to ship you the one container, and in the other case, it ships the other container, and the application would use it at runtime. So, everything will come together at runtime in the browser of the user, so you can switch everything. It's also possible to dynamically load containers via URL if you want something like that. Like, you get a server response like, load this container, and then the runtime would load this container and load some modules and instance, like, plugins or something you'd want to load dynamically. Okay. Well, I hope everyone has got his questions answered. But for those who didn't, of course, Tobias will be in his own private Zoom room where you can pick his brain on module federation or webpack in general, but trying to keep it on topic. So, Tobias, thanks a lot for this amazing talk. I'm looking forward to diving deeper into module federation. Thanks for having me.