Owning your Build-step – Owning your Code

Rate this content
Bookmark

Ever since JavaScript has become a language for writing applications, build tools and especially bundlers have been around. They solve the discrepancy between writing code that is easy to maintain and writing code that loads efficiently in a browser. But there are advantages to bundling JavaScript code that go well beyond the browser, from cloud functions to servers to command line tools.


RollupJS is special in that it was always designed from the ground up to be a general purpose bundler rather than a frontend specific tool. In this talk, we will have a look in what way other scenarios can profit from bundling. But more importantly, I will show you how RollupJS not only generates superior output in many situations, but how easy it is to tailor its output to custom requirements and non-standard scenarios. We will see how to patch up code, mock and replace dependencies, elegantly inject build information and control the chunk generation when code-splitting, all with a just few lines of code.

Lukas Taegert-Atkinson
Lukas Taegert-Atkinson
28 min
01 Jul, 2021

Comments

Sign in or register to post your comment.

Video Summary and Transcription

This Talk explores JavaScript code optimization using Rollup, showcasing examples of improved load times and reduced server size. It delves into Rollup customization and plugin development, demonstrating how to write plugins and remove code using hooks. The Talk also covers module code loading, advanced code control, and importing/emitting files with Rollup. Additionally, it highlights the adoption of Rollup's plugin system by other tools and introduces a self-made terminal used in the presentation.

Available in Español

1. Introduction to JavaScript Code Optimization

Short description:

Hello and thank you for tuning in to my session about how to improve your generated JavaScript code using Rollup. I want to tackle another question. Why are we using all these complex build pipelines and putting all this stuff around our JavaScript code? To give you two reasons, I'm going to show you some examples. The first example is an open-source, small Scrum Poker implementation. It can be run during development without bundling, and the load time is significantly reduced with bundling. The difference is due to the network requests and the waterfall effect of loading JavaScript files. Now let's consider a server scenario with a basic Apollo server set up in Docker. The Node 14 alpine image size is 116 megabytes.

Hello and thank you for tuning in to my session about how to improve your generated JavaScript code using Rollup. Before I want to go into that part, actually, I want to tackle another question.

The question is like the basic question of JavaScript. Why are we using all these complex build pipelines and putting all this stuff around our JavaScript code? To give you two reasons, I'm going to show you some examples first.

The first example is something we built at our company. It is open-source, a small Scrum Poker implementation. You might want to give it a try if you want to. The important part of this, why I chose it, is because with this application, it is possible to run it during development without bundling. I did some measurements like cranking my browser to a really slow network setting for a small mobile phone. The times I got is without bundling it takes around 18.3 seconds to load, but with bundling, it takes seven and a half seconds. Now, I want to say the only change between those numbers was the bundling. There's not a lot less code on the right side. There's no compression, no minification. Why is that, actually? Why is there this difference in numbers? The difference you can see, actually, in these two pictures. It's these green staircase steps here, or it's usually called a waterfall here. What's happening is, it's starting, these are all JavaScript files, starting with the first file, and then it sees that there's some imports in the file. Once it's discovered those imports, it sends the network request, okay, give me those more files. It needs to load those files, pass those files, and discover some more imports, and so on. Between all those steps here, there's, basically, network request back and forth, and this is adding up. Of course, what bundling is doing, it avoids the waterfall. Now the question is, okay, this is a web application, I know what we're doing here. What about our situations? Let's say we are looking at a server. This time, I have a a very basic Apollo server set up here, just taken from the website. This time, I'm not going to give you some made up numbers. This time, we're going to do it live. I'm going to build a server in Docker. To give you a reference point here, I have a small terminal here hooked up that is running commands on my machine. We're going to build a Node 14 alpine image. This is 116 megabytes. Maybe you want to remember this number.

2. Dockerfile Setup and Server Size Reduction

Short description:

We have two Docker files prepared here. The first one is a traditional setup, copying package files, installing production dependencies, and copying the server file. The second Dockerfile uses rollup to reduce the server size from 18MB to 4MB by removing unnecessary files in node modules. This reduces startup time and is beneficial for Cloud Functions and command line tools.

We have two Docker files prepared here. The first one I'm going to run is a very traditional setup. What we're doing is we're copying the package files. We are installing the production dependencies. We are copying the server file. This is this file. There's a small wrapper here. This wrapper is just there because it starts a small timer. The timer is stopped once the server says it's fully functional. That's just the measurements.

I'm just running a command that will also immediately start the server to get some time. What I'm doing here is I'm also copying everything over to another image, just the folder we just created. The reason is that during this year, a lot of caches are created, and this saves another two to three megabytes. I'm doing this now. This will just take a moment because all of it is cached right now. The first you see, you remember, it was 160 megabytes before, now it's 134 megabytes, so an 18 megabyte server basically. Startup time was nearly 300 milliseconds.

Now I've got a second Dockerfile here, which is this one. So what this one is doing is it's basically running rollup here, taking the server as an entry point and being naughty, just overriding it again, and there are three plugins, Node.resolve, CommonJS and JSON, which are necessary for node compatibility and we are creating CommonJS file, and that's all there is. And then we are basically copying just the created artifact over, and when I'm doing this, let's see what the numbers are now. You see, it's 120 megabytes, so the 18 megabyte server just became a four megabyte server.

So why is that? That's because there's really a lot of unnecessary stuff in your node modules. This is TypeScript types, this is test files, documentation, who knows what unneeded utilities. So this is maybe not as relevant if you say, okay, 130 versus 120, but this was a really basic setup. So this keeps adding up, the bigger your server becomes, and of course, startup time was 171 milliseconds, so it's nearly half the startup time. And this is, again, the same reason that you saw before. You are reducing the waterfall time. So we are seeing this reduces the size and the startup time, so servers are maybe not that important, but Cloud Functions definitely are. Those Cloud Functions really need quick startup time or also command line tools. So another question, why would you want to use rollup for this? So there are very good alternative choices.

3. Rollup Customization and Plugin Development

Short description:

Rollup is designed to be agnostic to the target environment, allowing for customization. It works well for the browser by default, but plugins are needed for other environments. Rollup's native format, ES modules, enables efficient bundling for libraries. Taking control of the code is easy with Rollup. In the first example, we import information from a file and want to customize it during production. To achieve this, we write a plugin and use the load hook.

I'm not going to say they are bad because they aren't. What is special about rollup is that rollup is designed from the ground up to be agnostic to the target environment. So you can customize it in any way you like. You can build for the browser, for note, for demo, whatever you want.

This also means that you usually need to add some plugins to customize it. So by default, it works quite well for the browser, but you definitely need some plugins for note. You also get a nice choice of output formats like common jazz ES modules versus various other forms like AMD and ES modules is actually special because that's rollup's native format, which means that if you bundle to ES modules and take the output and bundle it again, it just doesn't change anymore. It doesn't get any bigger, which means this is why rollup is actually used for libraries a lot. Because here, you don't want to have the runtime dependencies all the time. And also, we have very good dead code elimination, even though it wasn't important the previous examples.

So, I wanted to say we want to actually do stuff with our code. So we know why we want to bundle, the question is, of course, the how. And what I want to pattern I keep seeing here with rollup is that sometimes, even though they're very good plugins, you basically want to take things into your own hands. And you can definitely can because it's really easy.

So if you don't know what to do during the talk, you could actually join in right now because this is hosted live here. So you can go to lucastagger.github.io slash DevOps minus JS, if you want to start the current slide is hash seven. And if you aren't fast enough to type this, this URL will be on the top of the next two slides. So let's get started here. So the first example, so here's the I was talking about the first example I want to show you is a very common situation. You have a main file here, which is importing information from a file build JS. And in this case, it's just importing a string, which is telling us okay, this is a development built. But there could be more information here. Let's say there could be a different server doing productions or this could be localhost during development, but I don't know what server during production or alternative information. And what you want to do is basically you want to have the unchanged setup during development, but during production, we want to change this information. So let's actually write a plugin. So how do you do that? So, and as you can see, my setup is actually running rollup live while I'm typing, the output is here on the bottom right. So plugins are an array of objects. And so a plugin exposes several hooks that rollup can hook into. And the hook we are going to use for the first example is the load hook. The load hook receives an ID, which is an identifier of the file, and this is usually the path of the file on disk.

4. Writing a Plugin for ID slash build.js

Short description:

If the ID is slash build.js, we want to return something else, specifically export const type equals production. This is how simple it is to write a plugin.

So what we want to do is, if the ID is slash build.js, then we want to return something else. We want to return, so it needs to be, export something, export const type equals, and maybe use a new line that you can keep reading what I'm writing. Yeah. Production. And here we are, we just wrote our first plugin. So now in the output, it's const type equals production. And that's how simple it is. By the way, if you're following this live, again, the URLs here on top, you can just click the button here on the lower left to show you where we want to go to.

5. Removing Code with the Transform Hook

Short description:

In this example, we're using the transform hook to remove specific lines of code based on a custom syntax. By using a regular expression and the replace method, we can easily remove the targeted lines. This removal is not just cosmetic; it is integrated into Rollup's analysis and allows for efficient tree-shaking. Additionally, we briefly discussed the load and transform hooks and their role in the module lifecycle.

Okay. So this was loading stuff. Let's go to another example. So this time we're going to invent some syntax we want to use. So let's say we want to have special logging checks and so on during development that you want to remove during production. So the syntax we are making up here is that any time we precede a line with this comment, remove, then we want to remove everything up to the end of the line.

The hook we are using this time is the transform hook. The transform hook actually has two arguments. The first argument is the code of the module. The second would be the ID, but our current transformation doesn't depend on the individual module. How do you do this? Easiest would be actually to use a regular expression. We're just going to use code, replace, and replace can have a regular expression as its first argument, which is going to be a global expression. We want to replace all occurrences with an empty string. What do we need to put in here? It's slash star remove star slash. You see, I already removed my comment, but that's not what I wanted. We want to remove until the end of the line, so we need several characters which are not a line break. This is that, until the line break. And here we go. We just removed the second line. I could copy this to the first line. It would also be removed. And also note that this is not just some cosmetic removal of code. This is actually fully integrated into Rollup's analysis. So if I say I have some constant foo that I'm also including here in the logging and now I'm adding the comment here, the foo will also be recognized as unused and tree-shaking will just get rid of this in our build.

So we've seen two hooks so far, load and transform. To give you an idea how this goes into the wider picture, let's have a high level view on the lifecycle of a module. So the situation we have here, we have a module slash main.js and it just contains for now an import statement. So what does Rollup do with this code? The first thing is it will take this import statement and pass it to the resolve ID hook. You didn't see this one so far. We are going to use it in the next example.

6. Resolve ID Hook and Import Source

Short description:

The resolve ID hook is called on each plugin that implements it, and the second plugin can provide a specific import source for a given ID. The default algorithm appends the relative source to the directory of the importer.

And the resolve ID hook has two arguments. The first is the import source exactly as it is written here. And the second is the ID of the importing module. Now, this hook is then called on each plugin that implements it until one of the plugins answers it. So in this case, let's say the second plugin says, oh, I know what it means. If someone from slash major imports dot slash foo, then they want slash foo.js. Which is not surprising. And actually something that core would also have done because there's a default algorithm that will just do that. Take the directory of the importer and append the relative source here.

7. Module Code Loading and Transformation

Short description:

In the load hook, we get the code of the module by passing the discovered ID. The load hook checks each plugin, and the first one that implements it and returns something provides us with the code. The transform hook allows plugins to perform code transformations. We can replace arguments or inject imports. Another example is changing file resolution by using the resolve function in the context. It returns an object with the resolved ID.

Okay. So then we need to get the code of the module and this is done by the load hook. So the ID we just discovered, we are passing now on to the load hook. The load hook will again be checking each of the plug-ins. And the first plugin that implements it and also returns something will be the one to give us the code. And since in this example, no plugin implements it, we have a default implementation, which is assumed this is the path to a file and try to load it from the file system. So we are getting this code here now.

And then there's the transfer hook. Now that we basically have the first glimpse of the code. Each plugin has the chance to do some transformations on the code. This is intended for plugins like Babel, like transforming code, but any other code transformation is possible. Again, we have two parameters, the code and the ID, and it goes through all the plugins. The first might be replacing the argument here. The second might be injecting an import. And now that we have another import, it basically starts back from the top, and the cycle continues. This is not all plugin hooks that we have enrolled. There's actually quite a few more, but these will be the ones you probably want to use the most.

So now let's do another example. So this time we are revisiting our first example. We want to replace some build time information with production information. But this time we want to have two files which already exist on disk. So we could again use the load hook and just read this file in the load hook, but maybe it would be more would be nicer to actually change the file resolution. So each time this file is imported, we want to actually import this file. So it should be something like if ID equals equals equals log minus dev JS, then return slash log minus prod dot JS. Okay, this of course doesn't work because we don't have the ID. We only have the source and importer. So what we would like to know is, roll up, how would you resolve this source and importer to an ID? And for that we have context functions. So each plugin hook is called with a special this context containing several functions. This time we want to use this resolve. And since this is an asynchronous function, we want to evade it and it will return an object with an ID property.

8. Advanced Code Control and Self-Referencing Bundles

Short description:

Now let's take it to the next level by completely removing the file system dependency. We can replace information using a virtual file called 'build'. We resolve the build ID and handle the error of a missing file system. Finally, we explore the concept of a self-referencing bundle that knows its own files.

Okay. Now I'm not going to close this one because as soon as I write the closing parent piece, this will be an infinite loop because calling this resolve will actually call all plugins unless I'm passing a special argument, which is skip self two. And yes. And here we go. So the development build was this, the production build is now log a log and it's now replacing this file with the other one.

Okay. Another example. So taking this to the next level means we can actually completely without the file system. Up to now, the IDs were like parts of files, but you're not limited to this. So for the last version of this, we are going to replace information. Just, we're going to use a completely virtual file. We call it build without anything around this. So now using what we just learned, we can actually, we need to do two things. So it's already warning here that build cannot be resolved. So first of all, we need to resolve it and resolving is just telling Rollup, okay, it's totally fine. This is an ID. So what we're doing is just if source equals equals build, then just return the source again. Okay. And the error changed. Now this time it's loading this, and because we are using the browser built here on the presentation, it's complaining about the missing file system. So we're doing the same here. If the ID equals equals build, then just return export. Oh, okay. New line. And equals prod. And here we are. Our first completely virtual module.

And as a very last example of what you can do to basically take full control of your code, patch problems, and do any kind of transformation, like this time we want to do a self-referencing bundle. So we want a bundle that knows what files are in the bundle in the JavaScript code. And we're using a trick here.

9. Importing and Emitting Files with Rollup

Short description:

We're importing something from a file, from a file, slash build.js, and using it as an external import. Two files are generated due to the dynamic import statement, indicating lazy loading. We emit the file using the context helper 'emit file' with type 'asset', file name 'build.js', and an empty source. The generated bundle lists the names and sizes of the files. There's much more you can do with Rollup, including building your own plugins or using high-level tooling like Stencil, VEET, and WMR. For libraries, consider MicroBundle or TSDex. Thank you for staying with me and feel free to explore the website for examples.

So we're importing something from a file, from a file, slash build.js, and we're actually not implementing this file right now, but rather we are using this to indicate it's external. And you see already, it is just capped as an import here in the resulting files. Also, you see that there are actually two files generated. The reason is the dynamic import statement here, which Rolab interprets as you want to have some lazy loading here, so there will be a different chunk created just for this one.

And, okay, this is what we're doing here is we're emitting this file. So there's one context helper, emit file, which allows you to add files to the bundle. And an edit file has a type. The type is acid. You could also use chunk, but that has some limitations on how it can be used, so acid is usually what you want for arbitrary files. We need a file name. The file name should be build.js, and we need some source. And now we actually created an empty as in because time is running short, we're just using the shortcut here.

So what this one's doing is basically taking the bundle object, you get to the generate bundle hook, and it's just listing the names and the size of the file. So right now main.js it's 90 bytes and this one is 30 bytes, and if I were to change stuff here, you can actually see the number here changing life. So now it's 36 bytes and wrapping up the talk now. Really there's so much more you could do. So I recommend have a look at the website of Rollup telling you how to build your own plugins. And there's actually, if you don't want to go that route, there's also some high level tooling built around Rollup. So just to mention a few that's stencil for web components, VEET, which comes from the VUE ecosystem and it's a very nice development tool without bundling during development and with a prebuilt Rollup step. And WMR is very similar from the Preact community. And for libraries I can recommend to have a look at MicroBundle, which is also more from the Preact universe for zero configuration libraries or TSDex for TypeScript libraries. And with that, I want to conclude my talk. Thank you for staying with me. Just remember you can access the website locally on your browser if you do this, if you want to play with examples and thank you very much.

Hi, yeah. Nice to be here. Excellent to have you. So let's take a look at the results to your question. What is your experience with Rollup plugins? And while... Maybe I overdid it with a number of options."

10. Rollup Usage and Plugin Adoption

Short description:

I'm most surprised about the 11% because you can use Rollup without plugins, but then it's kind of limiting what you can do. I have great hopes for the 18% and I was doing the talk mostly for the 18%. There are new tools like VEET and WMR which are actually adopting Rollup's plugin system right now, so that you can basically use your plugin knowledge in other tools. Lots of love to the lower options with even though it's only 2% each.

Okay, so if I could have dropped the first questions I would have expected the majority not having used Rollup because it's kind of a niche tool and I know that. The interesting part is further down, actually I'm most surprised about the 11% because you can use Rollup without plugins, but then it's kind of limiting what you can do. I have great hopes for the 18% and I was doing the talk mostly for the 18%, never wrote a plugin but would consider trying it out because there is like kind of a meta layer for Rollup plugins that is emerging. So there are new tools like VEET and WMR which are actually adopting Rollup's plugin system right now, so that you can basically use your plugin knowledge in other tools. Lots of love to the lower options with even though it's only 2% each. I have to say that I chose the Chuck Norris option myself and I feel there's many other Chuck Nori at this conference so. I know you are.

11. Self-Made Terminal and GitHub Repo

Short description:

The terminal used in the presentation is a self-made, open-source web server. It was created by the speaker and can be found on the GitHub repo. The terminal code is surprisingly concise, fitting on one screen. The speaker humorously refers to themselves as an overachiever.

I know you are. Okay, let's get to the most important question that we've received so far from the crowd, which is, what an amazing terminal for the presentation. What is it? I know the answer but I'll allow you to tell me. The problem is I started a little early preparing for the presentation and when you have too much time, you start doing things. So, I started writing a small web server and put it all into a reveal.js presentation. So, basically, the terminal is all self-made, but it's also open source. I think so, on the last slide of the talk there is actually the url for the GitHub repo, if I'm not mistaken, where you can actually look how it's done, it's actually surprisingly little. So, it fits in into one screen of code, the entire terminal on the back end side. So, you're what they call an overachiever. Yeah. Very cool. I hope not too many people from my company are here watching this and we're discovering what I did with my time.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Levelling up Monorepos with npm Workspaces
DevOps.js Conf 2022DevOps.js Conf 2022
33 min
Levelling up Monorepos with npm Workspaces
Top Content
Learn more about how to leverage the default features of npm workspaces to help you manage your monorepo project while also checking out some of the new npm cli features.
Automating All the Code & Testing Things with GitHub Actions
React Advanced Conference 2021React Advanced Conference 2021
19 min
Automating All the Code & Testing Things with GitHub Actions
Top Content
Code tasks like linting and testing are critical pieces of a developer’s workflow that help keep us sane like preventing syntax or style issues and hardening our core business logic. We’ll talk about how we can use GitHub Actions to automate these tasks and help keep our projects running smoothly.
Fine-tuning DevOps for People over Perfection
DevOps.js Conf 2022DevOps.js Conf 2022
33 min
Fine-tuning DevOps for People over Perfection
Top Content
Demand for DevOps has increased in recent years as more organizations adopt cloud native technologies. Complexity has also increased and a "zero to hero" mentality leaves many people chasing perfection and FOMO. This session focusses instead on why maybe we shouldn't adopt a technology practice and how sometimes teams can achieve the same results prioritizing people over ops automation & controls. Let's look at amounts of and fine-tuning everything as code, pull requests, DevSecOps, Monitoring and more to prioritize developer well-being over optimization perfection. It can be a valid decision to deploy less and sleep better. And finally we'll examine how manual practice and discipline can be the key to superb products and experiences.
Why is CI so Damn Slow?
DevOps.js Conf 2022DevOps.js Conf 2022
27 min
Why is CI so Damn Slow?
We've all asked ourselves this while waiting an eternity for our CI job to finish. Slow CI not only wrecks developer productivity breaking our focus, it costs money in cloud computing fees, and wastes enormous amounts of electricity. Let’s take a dive into why this is the case and how we can solve it with better, faster tools.
The Zen of Yarn
DevOps.js Conf 2022DevOps.js Conf 2022
31 min
The Zen of Yarn
In the past years Yarn took a spot as one of the most common tools used to develop JavaScript projects, in no small part thanks to an opinionated set of guiding principles. But what are they? How do they apply to Yarn in practice? And just as important: how do they benefit you and your projects?
In this talk we won't dive into benchmarks or feature sets: instead, you'll learn how we approach Yarn’s development, how we explore new paths, how we keep our codebase healthy, and generally why we think Yarn will remain firmly set in our ecosystem for the years to come.

Workshops on related topic

Deploying React Native Apps in the Cloud
React Summit 2023React Summit 2023
88 min
Deploying React Native Apps in the Cloud
WorkshopFree
Cecelia Martinez
Cecelia Martinez
Deploying React Native apps manually on a local machine can be complex. The differences between Android and iOS require developers to use specific tools and processes for each platform, including hardware requirements for iOS. Manual deployments also make it difficult to manage signing credentials, environment configurations, track releases, and to collaborate as a team.
Appflow is the cloud mobile DevOps platform built by Ionic. Using a service like Appflow to build React Native apps not only provides access to powerful computing resources, it can simplify the deployment process by providing a centralized environment for managing and distributing your app to multiple platforms. This can save time and resources, enable collaboration, as well as improve the overall reliability and scalability of an app.
In this workshop, you’ll deploy a React Native application for delivery to Android and iOS test devices using Appflow. You’ll also learn the steps for publishing to Google Play and Apple App Stores. No previous experience with deploying native applications is required, and you’ll come away with a deeper understanding of the mobile deployment process and best practices for how to use a cloud mobile DevOps platform to ship quickly at scale.
MERN Stack Application Deployment in Kubernetes
DevOps.js Conf 2022DevOps.js Conf 2022
152 min
MERN Stack Application Deployment in Kubernetes
Workshop
Joel Lord
Joel Lord
Deploying and managing JavaScript applications in Kubernetes can get tricky. Especially when a database also has to be part of the deployment. MongoDB Atlas has made developers' lives much easier, however, how do you take a SaaS product and integrate it with your existing Kubernetes cluster? This is where the MongoDB Atlas Operator comes into play. In this workshop, the attendees will learn about how to create a MERN (MongoDB, Express, React, Node.js) application locally, and how to deploy everything into a Kubernetes cluster with the Atlas Operator.
Azure Static Web Apps (SWA) with Azure DevOps
DevOps.js Conf 2022DevOps.js Conf 2022
13 min
Azure Static Web Apps (SWA) with Azure DevOps
WorkshopFree
Juarez Barbosa Junior
Juarez Barbosa Junior
Azure Static Web Apps were launched earlier in 2021, and out of the box, they could integrate your existing repository and deploy your Static Web App from Azure DevOps. This workshop demonstrates how to publish an Azure Static Web App with Azure DevOps.
How to develop, build, and deploy Node.js microservices with Pulumi and Azure DevOps
DevOps.js Conf 2022DevOps.js Conf 2022
163 min
How to develop, build, and deploy Node.js microservices with Pulumi and Azure DevOps
Workshop
Alex Korzhikov
Andrew Reddikh
2 authors
The workshop gives a practical perspective of key principles needed to develop, build, and maintain a set of microservices in the Node.js stack. It covers specifics of creating isolated TypeScript services using the monorepo approach with lerna and yarn workspaces. The workshop includes an overview and a live exercise to create cloud environment with Pulumi framework and Azure services. The sessions fits the best developers who want to learn and practice build and deploy techniques using Azure stack and Pulumi for Node.js.