Lessons Learnt While Creating a New Framework on Top of React

Hello everyone, I'm thankful to React Advanced London team for inviting me to share what I have learned while creating a new full stack framework that allows developers to write React code to build frontends.

A bit about me, my name is Shyam Swaroop and I am the co-founder and CTO of Atrilabs, a company that is building an exciting new full stack framework. I was pursuing Masters from Columbia University, then I dropped out and started working on my own startup.

I have been using React for now more than a year, I was more of an Angular person before that. We can find the use of JSX beyond just web development. It has become one of the go-to choices for generating HTML, a use case very similar to SSG, SSR, or you can say a replacement for template engines. Some projects have extended grammar of languages like Go and Python to include JSX.

React goes beyond just front-end development. We are mixing plain content in markdown files with react code for writing content. This has given rise to frameworks that specialize in specific tasks like documentation. A popular example would be DocuSource. React Native was used to create native mobile apps for Androids and iOS. There are few frameworks that can be used to create native Windows or MacOS apps using React. React Reconciler can be adapted for rendering UI on any device. I recently came across a very interesting use case that does not involve rendering UI, but rather synthesizing music by writing React code. I remember the power I felt when I learned React or Bison, where you just write a context-free grammar and it generates the parser. Since I knew how to create parsers easily, I was able to innovate fast in many scenarios.

This is how I feel now with web development, because now that I know the React and tools to handle a React code base, I can easily create a very specialized framework that can enhance the developer productivity of a web developer team.

We have seen quite a few frameworks on top of React and all of these innovate and differentiate in their offerings for developers. It wasn't easy to reach this comfort level. When I decided to create a full stack framework, it took me more than a month to just understand the tooling ecosystem around React or JavaScript type script in general. This was even harder because I had to dive a bit into the history of JavaScript type script and how they came to be. I landed upon React's design philosophy which motivated me to write down philosophies for the framework.

It has really helped me in deciding two major categories of tools that our framework should use. Those are bundler and transpiler. More on this later. Once we had pinned these down, then only we can move to development of the framework. When we think of a framework, we quickly start to think that they provide abstraction, helper functions, components, server side rendering, server side generation, etc. We take many other things for granted. For example, how are we handling different JavaScript syntax and type script all in one code base? How are we able to import non-JavaScript typescript files in JavaScript typescript files? For example, we import image files as string URLs, and VG as React components, etc. These have become the de facto standards for a good developer experience in any framework.

This is made possible by tools like compilers, transpilers, bundlers, formatters, and linters. Rendering on the server side is a common requirement for most of the frameworks, if not all. Hence, our framework has to support React in both browsers and Node.js. In other words, it becomes crucial to realize the difference between different module types, especially ECMAScript and common.js. When we think about what bundler and build tools to use, we mostly think in terms of how fast they run. But as a framework creator, you might have to think more about your requirements. For example, if the developer's experience entails code mods, then you might prefer something like Babel over ESBuild because Babel exposes AST to load as plugins, while ESBuild doesn't. We're using Webpack and Babel together at a 3 framework.

A 3 framework allows developers to write plugins etc. where JavaScript modules are treated as different resources. We needed a way so that one resource can point to another resource. One way could have been that you write the path to a module you want to point as in the first statement you see here. It has several drawbacks. While the developer is writing or typing the code, the developer won't get any intellisense and no checks at build time. To circumvent these, we are using a feature from Webpack called resource query that we can add at the end of the import path as you can see in the second example. In the second example we are able to offer good intellisense and build time checks.

Once we have some idea about tools and the ecosystem, going through a few good codebases or frameworks helped us a lot. It speeded up our development where we are able to focus more on our unique offerings than the mundane ones. The best starting point was going through the createreactapp repository, and that's what we are going to look at in detail next. Looking at NextGIS repository is also a good idea.

Before we look into the code of createreactapp, I would like to classify the packages in createreactapp into three categories. Packages that are used to build a CRE application. These are the most important packages to go through. There are a few packages to configure the tools such as transpilers, for example, Babel and the linter such as ESLint. Finally, the packages to generate the initial scaffolding when we run something like NPX Create React App.

An important question is how are we able to insert non JavaScript modules like CSS, SVG, PNG, etc? We need to configure multiple tools to actually be able to achieve this in our We need to add types for non JavaScript files and we need to add types for Node.js namespace. We can do this by creating a TypeScript type declaration file and including it in our tsconfig.json file. Here we see that we are declaring model types for reports sourced and matching different globe patterns. Let's look at the highlighted part. This informs tools like eslint in our code editors that whenever the source string ends with svg, treat the source module as exporting a default string value that will contain the asset url and a React component named export which will be of type react.functional component. We also need to handle these imports in our bundler in our case webpack. More on how to configure webpack to achieve this in a later section.

As mentioned before we use the resource query feature from webpack.export to point to a module. Hence we had to add one more module definition here. Whenever an import source path ends with a question mark id. It will be treated as a module that default exports a string. The entry point to understanding the corrupts of everything that happens when you actually do a build during development or production is the webpack.config.js file. It can be found in React scripts package in create-react-app code base. The create-react-app uses many loaders that run one after the other to process CSS, as we can see here. The webpack will start from a post CSS loader that provides support for mixins, transpile upcoming CSS syntax, and inline images. It has some plugins that help insert vendor prefixes in your CSS. Webpack can handle JavaScript imports by default, but it cannot handle import statements in CSS. For example, when we use import directive in CSS or the URL function in CSS, it uses the CSS loader to resolve them. If you decide to use Sass or CSS, you might have to add a loader to preprocess it. The create react app uses Sass loader. See, we can see that over here, we go into Sass loader.

Cid apps handle SVGs using a third-party library called SVGR. This is different than handling other assets which are directly handled by webpack like this. Here we can see side effects. What does this mean? How does it affect the final bundle? This brings us to the next topic dead code elimination or pre-shaking. The bundle optimization techniques such as dead code elimination and mangling are activated by default in production mode. If we are not in production mode but we still want to have any of these optimization benefits, then we will have to import optimization plugins on our own.

Pre-shaking eliminates unused functions from across the world by starting at the entry point and only including functions that may be executed. React's higher order components make it harder for Tercer to pre-shake dead code. For pre-shaking to work, we have to use ES 2015 or ES6 module syntax that is import and export. We have to ensure that no compilers or loaders or plugins transform our ES6 and above syntax code into CommonJS module. Just as a precaution, if you are using Babel and you use Babel's Preset env, you might have to configure it so that it does not convert the ES6 modules to CommonJS which is its default behavior.

Coming back to where we left in the code, the Tercive will detect CSS imports as unused, hence it will try to remove them. To prevent it we need to set side effects field in Webpack as true. Let's look at how Tree Seeking works in Webpack with an example. In this example, the index file imports wrap-button, but does not use it. Webpack's Tercer plugin, tries to figure out if it can drop some code from this file. But it might not be able to figure it out if button-wrapper function call has some side effects or not. A side effect might be something like changing a value on window object or any other global variable. The highlighted code will be eliminated once we have added a comment like this, which says Pure. But there are still questions with the imports which need to be included or evaluated because they could contain side effects. In other words Webpack will traverse the dependencies in this module and figure out if imported modules have any side effects. This will hurt bundle time and bundle size. One way to prevent this is by letting Webpack know that no module in this package has any side effects by setting side effects field in package.json as false.

Imagine we are developing a dialog box that opens up after we click through a few buttons. We have to change the app's state to finally open that dialog box. It becomes time-consuming and annoying for frontend developers to open the dialog box after every change they make to that dialog box component. The solution is to somehow replace the module containing the dialog box component during runtime and this is commonly referred to as hot module replacement or HMR. In React's context, a question arises. How do we maintain the state of the component after the module containing this component has been replaced? React's Fast Refresh plugin handles reapplying the state of the app once the module has reloaded.

How do we maintain the state of the component after the module containing this component has been replaced? React's Fast Refresh plugin handles reapplying the state of the app once the module has reloaded. It needs to inject some code into each module to accomplish this. We use the React Refresh Babel plugin to actually handle this for us.

How do we decide whether the changes made in the module should trigger a full-page reload or a hot module replacement? To decide whether to hot replace the module or full-page reload, we can implement some checks on the module such as all the imports from the module must be a React component for the module to be eligible for HMR. Thankfully implementing HR for React is quite easy with bundlers like Webpack. Module bundlers like Webpack and Ruler have exposed HMR API that can be implemented on a module. Let's look into the code.

Here we are adding a Babel loader that will run for all JavaScript and TypeScript modules outside Node modules. And the second and the final step is to include the React refresh Webpack plugin in the Webpack config. How do we do HMR for CSS and other similar files? During development, a plugin can replace CSS files with JavaScript modules. This makes it possible to hot reload the JavaScript modules in order to emulate HMR for the CSS files. In webpack, we can use tile loader to achieve this. If you are interested in creating a full-stack framework, you can check out our open source repo here. You can find me on social with this handle.