Understanding JavaScript Compilation

Rate this content
Bookmark

Compiling (or transpiling) modern JavaScript to older ECMAScript-compatible versions has allowed the web to evolve incredibly fast while making us happier developers. But how does the compilation process work? How can we write our own Babel plugins? Let's find out!

8 min
18 Feb, 2022

Video Summary and Transcription

Today's Talk explores the concepts of compilation and transpilation in JavaScript, highlighting the three steps involved: parsing, code transformation, and code generation. The process of traversing an abstract syntax tree is discussed, with examples of tools like Babel and ESLint that can be used for code transformations. Code transformation tools and techniques, such as CodeShift and T-TypeScript plugin, are introduced, emphasizing the power of abstract syntax trees and the visitor pattern. The summary concludes by encouraging further exploration of code refactoring and transformations using tools like Babel, TSLint, and astexplorer.net.

1. Understanding JavaScript Compilation

Short description:

Today, we are going to be talking about compilation, or I should say transpilation. In fact, when talking about compilation, we refer to that process that takes a source code as an input and produces binary code or bytecode as an output. The whole compilation phase takes place in three different steps. Parsing, code transformation, and code generation.

♪♪♪ Hello, everyone, and welcome to my talk, Understanding JavaScript Compilation. Before we start, let me introduce myself briefly. I'm Michele Riva, Senior Architect at NearForm, author of Real World Next.js book, Google Developer Expert and Microsoft MVP.

Today, we are going to be talking about compilation, or I should say transpilation. In fact, when talking about compilation, we refer to that process that takes a source code as an input and produces binary code or bytecode as an output. In JavaScript, we are more likely to adopt transpilation. An example is the REST script compiler, which takes correct REST script code as an input and produces proper JavaScript code as an output. So from one source code to another. We can do the same with Babel, for example. In that example, we have the optional chaining and nullish operators, which are not supported on older runtimes and browsers. We want to make this code more compatible, so we take this source code, we feed the Babel compiler, and we get the following code as an output which is still valid JavaScript source code, which is more compatible with older runtimes.

The whole compilation phase takes place in three different steps. Parsing, code transformation, and code generation. Let's break them down. Step number one for parsing is tokenization. So we have var foo equals 10, and we divide it into separate tokens, var foo equals 10. And then we can create a parse tree. And here we can see the structure of our program and understand if it's correct or not. And the next step would be abstract syntax tree creation. So it's quite difficult to understand from these slides, so let me go on the other one. Okay, so this is how it's presented. We have variable declarations at the top. And again, JavaScript can create multiple variables in a row. In that case, we create just one variable. So, we will be ending up having just one variable declaration. So, var foo equals 10, it's the variable declaration. We can break it down to all the variable declarations we have. In that case, just one. So, variable declarator, where we declare that foo, it's equal to 10. We can break down the variable declarator again in two different parts, the identifier, foo, and numeric literal, which is 10.

2. Understanding Abstract Syntax Tree Traversal

Short description:

If we were writing like foo equals true, we would have wrote down the boolean literal instead of the numerical one. When reading an abstract syntax tree, we call this process traversing the tree. With Babel, we can transform variable declarations from var to let. The same concept applies to other tools like Prettier, ESLint, Babel, GIS, CodeShift. With ESLint, we can report and fix errors related to template literals.

If we were writing like foo equals true, we would have wrote down the boolean literal instead of the numerical one, or strings or whatever, just for you to understand. While it's quite easy to understand this tree, I guess, this one, it's a bit more challenging to deal with.

When reading an abstract syntax tree, we call this process traversing the tree. It's quite complex on, of course, complex programs. But luckily, many tools such as Babel, ESLint, Pretier, provides us a nice interface for traversing the tree which is called the visitor pattern.

In that case, with Babel, we're saying, okay, dear Babel, please give me all the variable declarations or the identifiers, all the numerical literals, and we just print them to the console. Once we get, for example, all the variable declarations, we can start transforming them. So that's the case. We say, okay, please give me all the variable declarations inside of our code. And if the kind of the variable is a var, just transform it into a let, which is not wise. I'm not suggesting you're doing that, but it's a very straightforward example. So that makes things easier for us. So that said, on the left, we have our input var foo equals 10. And on the right, we have our output let foo equals 10. As you can see, on line two, we have constant and we leave it untouched because it's not a variable, it's a, it's a const. So as you can see on the code, we are not saying if it's constant, please change it to let. So we are not doing that. As I said, the same concept applies to other tools, such as Prettier, ESLint, Babel, GIS, CodeShift. And we're gonna see them in details right now.

With ESLint, we have a similar approach. We say, okay, please, ESLint, give me all the template literals you can find in my code. And if I find one, I can report to ESLint an error, such as please do not use template literals. And I can also provide a fix for it. But in that case, if I have an expression inside my template literals, I can say, okay, I'm too lazy for that. So I'm just returning, I'm not fixing this. But if I don't have an expression, I can say, okay, so I don't have expression. I don't have any expression. So no dynamic data inside my template literals. So just change the literals into double quotes. So that's what we are doing now.

3. Code Transformation Tools and Techniques

Short description:

JS CodeShift is a powerful library built and maintained by Facebook for large-scale refactors. It allows you to find and replace identifiers, making it easier to refactor code. Another useful tool is the T-TypeScript plugin, which enables transformations like performing operations at compile time instead of runtime. By understanding abstract syntax trees and implementing the visitor pattern, you can achieve powerful code transformations. Tools like Babel, TSLint, YASLint, Recast, Pretier, and the website astexplorer.net can help you in this process. Thank you all for being here and feel free to reach out to me for further discussions on code refactoring and transformations.

And on the left, the input on the right, the output. JS CodeShift, it's another awesome example. It's a library built by Facebook and maintained by Facebook itself. And it's meant for big refactors, large scale refactors. And the concept is quite similar.

Here, we are saying, okay, please JS CodeShift, find all the identifiers. And if the identifier names contains the old name string, replace it with new name string. This is the case again for large-scale refactors. So we have like old name factory, which get refactored into new name factory.

There's also another awesome TypeScript plugin, which is called the T-TypeScript, which can help us implementing again nice transformations on the code. For example, we can say, if there's a binary operation such as one plus one, please report it to me. And then we can say, okay, this is a binary operation. We are adding two numbers. And instead of compiling to JavaScript, which will need to do that operation at runtime, do it at compile time. So that's the output. Const mysum originally was 10 plus 20. And as an output, we get const mysum equals 30. That's the power of knowing how to traverse an abstract syntax tree and implementing the visit of pattern.

And the same again applies with Babel, TSLint, YASLint, Recast, Pretier, JS CodeShift, and there's also an amazing website, which is called astexplorer.net that can really help you debugging your formulas and try to learn more about how to traverse a tree. So I really wanna thank you all for being here. It has been a real pleasure to talk at Node Congress, and these are my social media handlers, so if you want to reach out, I'd be glad to talk with you again about how to refactor code bases and making code transformations. That's what I love to do and to talk about. So thank you again.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Remix Conf Europe 2022Remix Conf Europe 2022
23 min
Scaling Up with Remix and Micro Frontends
Top Content
Do you have a large product built by many teams? Are you struggling to release often? Did your frontend turn into a massive unmaintainable monolith? If, like me, you’ve answered yes to any of those questions, this talk is for you! I’ll show you exactly how you can build a micro frontend architecture with Remix to solve those challenges.
Remix Conf Europe 2022Remix Conf Europe 2022
37 min
Full Stack Components
Top Content
Remix is a web framework that gives you the simple mental model of a Multi-Page App (MPA) but the power and capabilities of a Single-Page App (SPA). One of the big challenges of SPAs is network management resulting in a great deal of indirection and buggy code. This is especially noticeable in application state which Remix completely eliminates, but it's also an issue in individual components that communicate with a single-purpose backend endpoint (like a combobox search for example).
In this talk, Kent will demonstrate how Remix enables you to build complex UI components that are connected to a backend in the simplest and most powerful way you've ever seen. Leaving you time to chill with your family or whatever else you do for fun.
JSNation Live 2021JSNation Live 2021
29 min
Making JavaScript on WebAssembly Fast
Top Content
JavaScript in the browser runs many times faster than it did two decades ago. And that happened because the browser vendors spent that time working on intensive performance optimizations in their JavaScript engines.Because of this optimization work, JavaScript is now running in many places besides the browser. But there are still some environments where the JS engines can’t apply those optimizations in the right way to make things fast.We’re working to solve this, beginning a whole new wave of JavaScript optimization work. We’re improving JavaScript performance for entirely different environments, where different rules apply. And this is possible because of WebAssembly. In this talk, I'll explain how this all works and what's coming next.
React Summit 2023React Summit 2023
24 min
Debugging JS
As developers, we spend much of our time debugging apps - often code we didn't even write. Sadly, few developers have ever been taught how to approach debugging - it's something most of us learn through painful experience.  The good news is you _can_ learn how to debug effectively, and there's several key techniques and tools you can use for debugging JS and React apps.
Node Congress 2022Node Congress 2022
26 min
It's a Jungle Out There: What's Really Going on Inside Your Node_Modules Folder
Top Content
Do you know what’s really going on in your node_modules folder? Software supply chain attacks have exploded over the past 12 months and they’re only accelerating in 2022 and beyond. We’ll dive into examples of recent supply chain attacks and what concrete steps you can take to protect your team from this emerging threat.
You can check the slides for Feross' talk here.

Workshops on related topic

React Day Berlin 2022React Day Berlin 2022
86 min
Using CodeMirror to Build a JavaScript Editor with Linting and AutoComplete
Top Content
WorkshopFree
Using a library might seem easy at first glance, but how do you choose the right library? How do you upgrade an existing one? And how do you wade through the documentation to find what you want?
In this workshop, we’ll discuss all these finer points while going through a general example of building a code editor using CodeMirror in React. All while sharing some of the nuances our team learned about using this library and some problems we encountered.
Node Congress 2023Node Congress 2023
109 min
Node.js Masterclass
Workshop
Have you ever struggled with designing and structuring your Node.js applications? Building applications that are well organised, testable and extendable is not always easy. It can often turn out to be a lot more complicated than you expect it to be. In this live event Matteo will show you how he builds Node.js applications from scratch. You’ll learn how he approaches application design, and the philosophies that he applies to create modular, maintainable and effective applications.

Level: intermediate
TestJS Summit - January, 2021TestJS Summit - January, 2021
173 min
Testing Web Applications Using Cypress
WorkshopFree
This workshop will teach you the basics of writing useful end-to-end tests using Cypress Test Runner.
We will cover writing tests, covering every application feature, structuring tests, intercepting network requests, and setting up the backend data.
Anyone who knows JavaScript programming language and has NPM installed would be able to follow along.
Node Congress 2023Node Congress 2023
63 min
0 to Auth in an Hour Using NodeJS SDK
WorkshopFree
Passwordless authentication may seem complex, but it is simple to add it to any app using the right tool.
We will enhance a full-stack JS application (Node.JS backend + React frontend) to authenticate users with OAuth (social login) and One Time Passwords (email), including:- User authentication - Managing user interactions, returning session / refresh JWTs- Session management and validation - Storing the session for subsequent client requests, validating / refreshing sessions
At the end of the workshop, we will also touch on another approach to code authentication using frontend Descope Flows (drag-and-drop workflows), while keeping only session validation in the backend. With this, we will also show how easy it is to enable biometrics and other passwordless authentication methods.
Table of contents- A quick intro to core authentication concepts- Coding- Why passwordless matters
Prerequisites- IDE for your choice- Node 18 or higher
JSNation 2023JSNation 2023
104 min
Build and Deploy a Backend With Fastify & Platformatic
WorkshopFree
Platformatic allows you to rapidly develop GraphQL and REST APIs with minimal effort. The best part is that it also allows you to unleash the full potential of Node.js and Fastify whenever you need to. You can fully customise a Platformatic application by writing your own additional features and plugins. In the workshop, we’ll cover both our Open Source modules and our Cloud offering:- Platformatic OSS (open-source software) — Tools and libraries for rapidly building robust applications with Node.js (https://oss.platformatic.dev/).- Platformatic Cloud (currently in beta) — Our hosting platform that includes features such as preview apps, built-in metrics and integration with your Git flow (https://platformatic.dev/). 
In this workshop you'll learn how to develop APIs with Fastify and deploy them to the Platformatic Cloud.
JSNation Live 2021JSNation Live 2021
156 min
Building a Hyper Fast Web Server with Deno
WorkshopFree
Deno 1.9 introduced a new web server API that takes advantage of Hyper, a fast and correct HTTP implementation for Rust. Using this API instead of the std/http implementation increases performance and provides support for HTTP2. In this workshop, learn how to create a web server utilizing Hyper under the hood and boost the performance for your web apps.