Lessons Learnt While Creating a New Framework on Top of React

Hello, everyone. I'm thankful to React Advanced London team for inviting me to share what I have learned while creating a new full stack framework that allows developers to write React code to build frontends. A bit about me. My name is Shyam Swaroop and I am the cofounder and CTO for 3Labs, a company that is building an exciting new full stack framework. I was pursuing master's from Columbia University, then I dropped out and started working on my own startup. I have been using React for now more than a year. I was more of an Angular person before that. We can find the use of JSX beyond just web development. It has become one of the go-to choices for generating HTML, a use case very similar to SSG, SSR, or you can see a replacement for template engines. Some projects have extended the grammar of languages like Go and Python to include JSX. React goes beyond just frontend development. We are mixing plain content in markdown files with React code for writing content. This has given rise to frameworks that specialize in specific tasks like documentation. A popular example would be Docusaurus. React Native is used to create native mobile apps for Androids and iOS. There are a few frameworks that can be used to create native windows or macOS apps using React. React Reconciler can be adapted for rendering UI on any device. I recently came across a very interesting use case that does not involve rendering UI, but rather synthesizing music by writing React code. I remember the power I felt when I learned YACC or Bison, where you just write a context-free grammar and it generates the parser. Since I knew how to create parsers easily, I was able to innovate fast in many scenarios. This is how I feel now with web development, because now that I know the React and tools to handle a React code base, I can easily create a very specialized framework that can enhance the developer productivity of a web developer team. We have seen quite a few frameworks on top of React, and all of these innovate and differentiate in their offerings for developers. It wasn't easy to reach this comfort level. When I decided to create a full stack framework, it took me more than a month to just understand the tooling ecosystem around React or JavaScript TypeScript in general. This was even harder because I had to dive a bit into the history of JavaScript TypeScript and how they came to be. I landed upon React's design philosophy, which motivated me to write down philosophies for the Athery framework. It has really helped me in deciding two major categories of tools that our framework should use. Those are bundler and transpiler. More on this later. Once we had pinned these down, then only we can move to development of the framework. When we think of a framework, we quickly start to think that they provide abstraction, helper functions, components, server-side rendering, server-side generation, etc. We take many other things for granted. For example, how are they handling different JavaScript syntax and TypeScript all in one code base? How are we able to import non-JavaScript TypeScript files in JavaScript TypeScript files? For example, we import image files as string URL, as VG, as React component, etc. These have become the de facto standards for a good developer experience in any framework. This is made possible by tools like compilers, transpilers, bundlers, formatters, and linters. Rendering on the server-side is a common requirement for most of the frameworks, if not all. The JSR framework has to support React in both browsers and Node.js. In other words, it becomes crucial to realize the difference between different module types, especially ECMAScript and CommonJS. When we think about what bundler and build tools to use, we mostly think in terms of how fast they run. But as a framework creator, you might have to think more about your requirements. For example, if the developer's experience entails code mods, then you might prefer something like Babel over ESBuild, because Babel exposes AST to load as a plugin while ESBuild doesn't. We are using Webpack and Babel together at a 3 framework. A 3 framework allows developers to write plugins, etc. where JavaScript modules are treated as different resources. We needed a way so that one resource can point to another resource. One way could have been that you write the path to a module you want to point as in the first statement you see here. It has several drawbacks. While the developer is writing or typing the code, the developer won't get any intellisense and no checks at build time. To circumvent these, we are using a feature from Webpack called Resource Query that we can add at the end of the import path as you can see in the second example. In the second example, we are able to offer good intellisense and build time checks. Once we have some idea about tools and the ecosystem, going through a few good codebases of frameworks helped us a lot. It speed up our development where we are able to focus more on our unique offerings than the mundane ones. The best starting point was going through the Create React App repository and that's what we are going to look at in detail next. Looking at Next.js repository is also a good idea. Before we look into the code of Create React App, I would like to classify the packages in Create React App into three categories. Packages that are used to build a CRE application. These are the most important packages to go through. There are a few packages to configure the tools such as transpilers, for example, Babel and the linter such as ESLint. Finally the packages used to generate the initial skiffolding when we run something like npx Create React App. An important question is how are we able to insert non-JavaScript modules like CSS, SVG, PNG, etc. We need to configure multiple tools to actually be able to achieve this in our framework. We need to add types for non-JavaScript files, modules, and we need to add types for Node.js namespace. We can do this by creating a TypeScript type declaration file and including it in our tsconfig.json file. Here we see that we are declaring module types for both sources and matching different globe patterns. Let's look at the highlighted part. This informed tools like ESLint in our core editors that whenever the source string ends with SVG, treat the source module as exporting a default string value that will contain the asset URL and a React component named export which will be of type React.functionalComponent. We also need to handle these imports in our bundler in our case webpack. More on how to configure webpack to achieve this in a later section. As mentioned before, we use the resource query feature from webpack to point to a module. Hence we had to add one more module definition here. Whenever an import source path ends with a question mark ID, it will be treated as a module that default exports the string. The entry point to understanding the crux of everything that happens when you actually do a build during development or production is the webpack.config.js file. It can be found in React scripts package in create React app code base. The create React app uses many loaders that are run one after the other to process CSS as we can see here. The webpack will start from a post CSS loader that provides support for mixins, transpile upcoming CSS syntax, and inline images. It has some plugins that help insert vendor prefixes in your CSS. Webpack can handle JavaScript imports by default, but it cannot handle import statements in CSS. For example, when we use import directive in CSS or the URL function in CSS, it uses the CSS loader to resolve them. If you decide to use SAS or CSS, you might have to add a loader to preprocess it. The create React app uses SAS loader. So we can see that over here, we go into SAS loader. CRE apps handle SVGs using a third party library called SVGR. This is different than handling other assets, which are directly handled by webpack like this. Here we can see side effects. What does this mean? How does it affect the final bundle? This brings us to the next topic, dead code elimination or pre-shaking. The bundle optimization techniques such as dead code elimination and mangling are activated by default in production mode. If you are not in production mode, but we still want to have any of these optimization benefits, then we'll have to import optimization plugins on our own. Mangling eliminates unused functions from across the world by starting at the entry point and only including functions that may be executed. React's higher order components make it harder for Tercer to pre-shake dead code. For pre-shaking to work, we have to use ES2015 or ES6 module syntax, that is import and export. We have to ensure that no compilers or loaders or plugins transforms our ES6 and above syntax code into common JS module. Just as a precaution, if you are using Babel and you use Babel's preset env, you might have to configure it so that it does not convert the ES6 modules to common JS, which is its default behavior. Coming back to where we left in the code, the Tercer will detect CSS imports as unused. Hence, it will try to remove them. To prevent it, we need to set side effects field here in webpack config as true. Let's look at how tree shaking works in webpack with an example. In this example, the index file imports wrapped button but does not use it. Webpack's Tercer plugin tries to figure out if it can drop some code from this file but it might not be able to figure it out if button wrapper function call has some side effects or not. A side effect might be something like changing a value on window object or any other global variable. The highlighted code will be eliminated once we have added a comment like this which says pure. But there are still questions with the imports which need to be included or evaluated because they could contain side effects. In other words, webpack will traverse the dependencies in this module and figure out if the imported modules have any side effects. This will hurt bundle time and bundle size. One way to prevent this is by letting webpack know that no module in this package has any side effects by setting side effects field in package.json as false. This will prevent webpack from traversing further in the dependencies. Imagine we are developing a dialog box that opens up after we click through a few buttons. We have to change the app's state to finally open that dialog box. It becomes time consuming and annoying for frontend developers to open the dialog box after every change they make to that dialog box component. The solution is to somehow replace the module containing the dialog box component during runtime and this is commonly referred to as hot module replacement or HMR. In React's context, a question arises. How do we maintain the state of the component after the module containing this component has been replaced? React fast refresh plugin handles reapplying the state of the app once the module has reloaded. It needs to inject some code into each module to accomplish this. We use the React refresh beable plugin to actually handle this for us. How do we decide whether the changes made in the module should trigger a full page reload or a hot module replacement? To decide whether to hot replace the module or full page reload, we can implement some checks on the module such as all the imports from the module must be a React component for the module to be eligible for HMR. Thankfully implementing HR for React is quite easy with bundlers like Webpack. Module bundlers like Webpack and Roller have exposed HMR API that can be implemented on a module. Let's look into the code. Here we are adding a Babel loader that will run for all JavaScript and TypeScript modules outside node modules. And the second and the final step is to include the React refresh Webpack plugin in the Webpack config. How do we do HMR for CSS and other similar files? During development, a plugin can replace CSS files with JavaScript modules. This makes it possible to hot reload the JavaScript modules in order to emulate HMR for the CSS files. In Webpack, we can use style loader to achieve this. If you're interested in creating a full stack framework, you can check out our open source repo here. You can find me on social with this handle. Thanks for listening.