Lessons Learnt While Creating a New Framework on Top of React

Bookmark
Slides

React is so powerful that it can be used for more than frontend development. E.g. creating UI programmatically in the backend using a rule-based system or machine learning is another use case where React can serve as the best fit. Similarly, documentation earlier used to have only .md files but now it also includes .mdx file that contains React code. To fulfill such use cases, developers need to understand the internals of React and the tooling around it. Some great sources are popular codebases such as create-react-app, Next.js, etc. In this talk, I will share the lessons we learnt while creating a framework that achieves more than web development using React. Firstly, I will cover how create-react-app codebase is the best codebase to understand how webpack, babel, eslint, typescript etc. can be used alongside React. Secondly, I will share how Next.js teaches us to create a js framework where we can write backend and frontend in the same file and still be able to separate the two during calls. Thirdly, I will share how our framework supports plugins, i.e. the React code resides in multiple repositories but all of this code can share a single React runtime inside the browser. This is a very advanced use of React that cannot be achieved by code splitting using React.lazy. This talk will enable developers to use React for more than frontend development.

20 min
24 Oct, 2022

Comments

Sign in or register to post your comment.

AI Generated Video Summary

React is a versatile tool that goes beyond front-end development and can be used for synthesizing music. Creating a specialized framework with React requires understanding the tooling ecosystem and React's design philosophy. Examining good codebases like createreactapp can accelerate development. Handling side effects and module replacement can be achieved through techniques like dead code elimination and hot module replacement. Maintaining component state and implementing hot module replacement are important considerations for a React application.

1. Introduction to React and Its Applications

Short description:

Hello everyone, I'm thankful to React Advanced London team for inviting me to share what I have learned while creating a new full stack framework. A bit about me, my name is Shyam Swaroop and I am the co-founder and CTO of Atrilabs. I have been using React for now more than a year, and it has become one of the go-to choices for generating HTML. React goes beyond just front-end development. We are mixing plain content in markdown files with react code for writing content. React Native was used to create native mobile apps for Androids and iOS. React Reconciler can be adapted for rendering UI on any device. I recently came across a very interesting use case that does not involve rendering UI, but rather synthesizing music by writing React code.

Hello everyone, I'm thankful to React Advanced London team for inviting me to share what I have learned while creating a new full stack framework that allows developers to write React code to build frontends.

A bit about me, my name is Shyam Swaroop and I am the co-founder and CTO of Atrilabs, a company that is building an exciting new full stack framework. I was pursuing Masters from Columbia University, then I dropped out and started working on my own startup.

I have been using React for now more than a year, I was more of an Angular person before that. We can find the use of JSX beyond just web development. It has become one of the go-to choices for generating HTML, a use case very similar to SSG, SSR, or you can say a replacement for template engines. Some projects have extended grammar of languages like Go and Python to include JSX.

React goes beyond just front-end development. We are mixing plain content in markdown files with react code for writing content. This has given rise to frameworks that specialize in specific tasks like documentation. A popular example would be DocuSource. React Native was used to create native mobile apps for Androids and iOS. There are few frameworks that can be used to create native Windows or MacOS apps using React. React Reconciler can be adapted for rendering UI on any device. I recently came across a very interesting use case that does not involve rendering UI, but rather synthesizing music by writing React code. I remember the power I felt when I learned React or Bison, where you just write a context-free grammar and it generates the parser. Since I knew how to create parsers easily, I was able to innovate fast in many scenarios.

2. Creating a Specialized Framework with React

Short description:

Now that I know React and the tools to handle a React code base, I can easily create a specialized framework to enhance developer productivity. Understanding the tooling ecosystem and React's design philosophy were crucial in defining the framework's philosophies. Tools like compilers, transpilers, bundlers, and linters enable handling different JavaScript syntax and importing non-JavaScript typescript files. Supporting React in both browsers and Node.js requires realizing the difference between module types. Choosing the right bundler and build tools depends on your requirements, such as code mods. We use Webpack and Babel together, leveraging Webpack's resource query feature for better intellisense and build time checks.

This is how I feel now with web development, because now that I know the React and tools to handle a React code base, I can easily create a very specialized framework that can enhance the developer productivity of a web developer team.

We have seen quite a few frameworks on top of React and all of these innovate and differentiate in their offerings for developers. It wasn't easy to reach this comfort level. When I decided to create a full stack framework, it took me more than a month to just understand the tooling ecosystem around React or JavaScript type script in general. This was even harder because I had to dive a bit into the history of JavaScript type script and how they came to be. I landed upon React's design philosophy which motivated me to write down philosophies for the framework.

It has really helped me in deciding two major categories of tools that our framework should use. Those are bundler and transpiler. More on this later. Once we had pinned these down, then only we can move to development of the framework. When we think of a framework, we quickly start to think that they provide abstraction, helper functions, components, server side rendering, server side generation, etc. We take many other things for granted. For example, how are we handling different JavaScript syntax and type script all in one code base? How are we able to import non-JavaScript typescript files in JavaScript typescript files? For example, we import image files as string URLs, and VG as React components, etc. These have become the de facto standards for a good developer experience in any framework.

This is made possible by tools like compilers, transpilers, bundlers, formatters, and linters. Rendering on the server side is a common requirement for most of the frameworks, if not all. Hence, our framework has to support React in both browsers and Node.js. In other words, it becomes crucial to realize the difference between different module types, especially ECMAScript and common.js. When we think about what bundler and build tools to use, we mostly think in terms of how fast they run. But as a framework creator, you might have to think more about your requirements. For example, if the developer's experience entails code mods, then you might prefer something like Babel over ESBuild because Babel exposes AST to load as plugins, while ESBuild doesn't. We're using Webpack and Babel together at a 3 framework.

A 3 framework allows developers to write plugins etc. where JavaScript modules are treated as different resources. We needed a way so that one resource can point to another resource. One way could have been that you write the path to a module you want to point as in the first statement you see here. It has several drawbacks. While the developer is writing or typing the code, the developer won't get any intellisense and no checks at build time. To circumvent these, we are using a feature from Webpack called resource query that we can add at the end of the import path as you can see in the second example. In the second example we are able to offer good intellisense and build time checks.

3. Understanding the Create React App Codebase

Short description:

Going through good codebases or frameworks sped up our development. The createreactapp repository and NextGIS repository were helpful starting points. Classifying the packages in createreactapp into three categories: building a CRE application, configuring tools, and generating initial scaffolding. Configuring multiple tools to insert non-JavaScript modules like CSS, SVG, PNG. Handling imports in the bundler, webpack. Understanding the webpack.config.js file and the loaders used by create-react-app. The importance of adding a loader to handle import statements in CSS, such as Sass loader.

Once we have some idea about tools and the ecosystem, going through a few good codebases or frameworks helped us a lot. It speeded up our development where we are able to focus more on our unique offerings than the mundane ones. The best starting point was going through the createreactapp repository, and that's what we are going to look at in detail next. Looking at NextGIS repository is also a good idea.

Before we look into the code of createreactapp, I would like to classify the packages in createreactapp into three categories. Packages that are used to build a CRE application. These are the most important packages to go through. There are a few packages to configure the tools such as transpilers, for example, Babel and the linter such as ESLint. Finally, the packages to generate the initial scaffolding when we run something like NPX Create React App.

An important question is how are we able to insert non JavaScript modules like CSS, SVG, PNG, etc? We need to configure multiple tools to actually be able to achieve this in our We need to add types for non JavaScript files and we need to add types for Node.js namespace. We can do this by creating a TypeScript type declaration file and including it in our tsconfig.json file. Here we see that we are declaring model types for reports sourced and matching different globe patterns. Let's look at the highlighted part. This informs tools like eslint in our code editors that whenever the source string ends with svg, treat the source module as exporting a default string value that will contain the asset url and a React component named export which will be of type react.functional component. We also need to handle these imports in our bundler in our case webpack. More on how to configure webpack to achieve this in a later section.

As mentioned before we use the resource query feature from webpack.export to point to a module. Hence we had to add one more module definition here. Whenever an import source path ends with a question mark id. It will be treated as a module that default exports a string. The entry point to understanding the corrupts of everything that happens when you actually do a build during development or production is the webpack.config.js file. It can be found in React scripts package in create-react-app code base. The create-react-app uses many loaders that run one after the other to process CSS, as we can see here. The webpack will start from a post CSS loader that provides support for mixins, transpile upcoming CSS syntax, and inline images. It has some plugins that help insert vendor prefixes in your CSS. Webpack can handle JavaScript imports by default, but it cannot handle import statements in CSS. For example, when we use import directive in CSS or the URL function in CSS, it uses the CSS loader to resolve them. If you decide to use Sass or CSS, you might have to add a loader to preprocess it. The create react app uses Sass loader. See, we can see that over here, we go into Sass loader.

4. Handling Side Effects and Module Replacement

Short description:

Cid apps handle SVGs using a third-party library called SVGR. Dead code elimination or pre-shaking eliminates unused functions from across the world by starting at the entry point and only including functions that may be executed. To prevent Webpack from traversing further in the dependencies, let it know that no module in this package has any side effects by setting side effects field in package.json as false. Hot module replacement or HMR allows replacing the module containing the dialog box component during runtime and React's Fast Refresh plugin handles reapplying the state of the app once the module has reloaded.

Cid apps handle SVGs using a third-party library called SVGR. This is different than handling other assets which are directly handled by webpack like this. Here we can see side effects. What does this mean? How does it affect the final bundle? This brings us to the next topic dead code elimination or pre-shaking. The bundle optimization techniques such as dead code elimination and mangling are activated by default in production mode. If we are not in production mode but we still want to have any of these optimization benefits, then we will have to import optimization plugins on our own.

Pre-shaking eliminates unused functions from across the world by starting at the entry point and only including functions that may be executed. React's higher order components make it harder for Tercer to pre-shake dead code. For pre-shaking to work, we have to use ES 2015 or ES6 module syntax that is import and export. We have to ensure that no compilers or loaders or plugins transform our ES6 and above syntax code into CommonJS module. Just as a precaution, if you are using Babel and you use Babel's Preset env, you might have to configure it so that it does not convert the ES6 modules to CommonJS which is its default behavior.

Coming back to where we left in the code, the Tercive will detect CSS imports as unused, hence it will try to remove them. To prevent it we need to set side effects field in Webpack as true. Let's look at how Tree Seeking works in Webpack with an example. In this example, the index file imports wrap-button, but does not use it. Webpack's Tercer plugin, tries to figure out if it can drop some code from this file. But it might not be able to figure it out if button-wrapper function call has some side effects or not. A side effect might be something like changing a value on window object or any other global variable. The highlighted code will be eliminated once we have added a comment like this, which says Pure. But there are still questions with the imports which need to be included or evaluated because they could contain side effects. In other words Webpack will traverse the dependencies in this module and figure out if imported modules have any side effects. This will hurt bundle time and bundle size. One way to prevent this is by letting Webpack know that no module in this package has any side effects by setting side effects field in package.json as false.

Imagine we are developing a dialog box that opens up after we click through a few buttons. We have to change the app's state to finally open that dialog box. It becomes time-consuming and annoying for frontend developers to open the dialog box after every change they make to that dialog box component. The solution is to somehow replace the module containing the dialog box component during runtime and this is commonly referred to as hot module replacement or HMR. In React's context, a question arises. How do we maintain the state of the component after the module containing this component has been replaced? React's Fast Refresh plugin handles reapplying the state of the app once the module has reloaded.

5. Maintaining Component State and Implementing HMR

Short description:

React's Fast Refresh plugin handles reapplying the state of the app once the module has reloaded. We use the React Refresh Babel plugin to handle this. To decide whether to hot replace the module or full-page reload, we can implement checks such as requiring all imports from the module to be a React component. Implementing HMR for React is easy with bundlers like Webpack. Adding a Babel loader and including the React refresh Webpack plugin are the steps to enable HMR. HMR for CSS files can be achieved by replacing them with JavaScript modules using a plugin like tile loader in webpack.

How do we maintain the state of the component after the module containing this component has been replaced? React's Fast Refresh plugin handles reapplying the state of the app once the module has reloaded. It needs to inject some code into each module to accomplish this. We use the React Refresh Babel plugin to actually handle this for us.

How do we decide whether the changes made in the module should trigger a full-page reload or a hot module replacement? To decide whether to hot replace the module or full-page reload, we can implement some checks on the module such as all the imports from the module must be a React component for the module to be eligible for HMR. Thankfully implementing HR for React is quite easy with bundlers like Webpack. Module bundlers like Webpack and Ruler have exposed HMR API that can be implemented on a module. Let's look into the code.

Here we are adding a Babel loader that will run for all JavaScript and TypeScript modules outside Node modules. And the second and the final step is to include the React refresh Webpack plugin in the Webpack config. How do we do HMR for CSS and other similar files? During development, a plugin can replace CSS files with JavaScript modules. This makes it possible to hot reload the JavaScript modules in order to emulate HMR for the CSS files. In webpack, we can use tile loader to achieve this. If you are interested in creating a full-stack framework, you can check out our open source repo here. You can find me on social with this handle.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

React Summit Remote Edition 2021React Summit Remote Edition 2021
33 min
Building Better Websites with Remix
Remix is a new web framework from the creators of React Router that helps you build better, faster websites through a solid understanding of web fundamentals. Remix takes care of the heavy lifting like server rendering, code splitting, prefetching, and navigation and leaves you with the fun part: building something awesome!
React Advanced Conference 2022React Advanced Conference 2022
25 min
A Guide to React Rendering Behavior
React is a library for "rendering" UI from components, but many users find themselves confused about how React rendering actually works. What do terms like "rendering", "reconciliation", "Fibers", and "committing" actually mean? When do renders happen? How does Context affect rendering, and how do libraries like Redux cause updates? In this talk, we'll clear up the confusion and provide a solid foundation for understanding when, why, and how React renders. We'll look at: - What "rendering" actually is - How React queues renders and the standard rendering behavior - How keys and component types are used in rendering - Techniques for optimizing render performance - How context usage affects rendering behavior| - How external libraries tie into React rendering
JSNation 2022JSNation 2022
28 min
Full Stack Documentation
Interactive web-based tutorials have become a staple of front end frameworks, and it's easy to see why — developers love being able to try out new tools without the hassle of installing packages or cloning repos.
But in the age of full stack meta-frameworks like Next, Remix and SvelteKit, these tutorials only go so far. In this talk, we'll look at how we on the Svelte team are using cutting edge web technology to rethink how we teach each other the tools of our trade.
React Advanced Conference 2022React Advanced Conference 2022
30 min
Using useEffect Effectively
Can useEffect affect your codebase negatively? From fetching data to fighting with imperative APIs, side effects are one of the biggest sources of frustration in web app development. And let’s be honest, putting everything in useEffect hooks doesn’t help much. In this talk, we'll demystify the useEffect hook and get a better understanding of when (and when not) to use it, as well as discover how declarative effects can make effect management more maintainable in even the most complex React apps.
React Summit 2022React Summit 2022
20 min
Routing in React 18 and Beyond
Concurrent React and Server Components are changing the way we think about routing, rendering, and fetching in web applications. Next.js recently shared part of its vision to help developers adopt these new React features and take advantage of the benefits they unlock.
In this talk, we’ll explore the past, present and future of routing in front-end applications and discuss how new features in React and Next.js can help us architect more performant and feature-rich applications.
React Advanced Conference 2021React Advanced Conference 2021
27 min
(Easier) Interactive Data Visualization in React
If you’re building a dashboard, analytics platform, or any web app where you need to give your users insight into their data, you need beautiful, custom, interactive data visualizations in your React app. But building visualizations hand with a low-level library like D3 can be a huge headache, involving lots of wheel-reinventing. In this talk, we’ll see how data viz development can get so much easier thanks to tools like Plot, a high-level dataviz library for quick
&
easy charting, and Observable, a reactive dataviz prototyping environment, both from the creator of D3. Through live coding examples we’ll explore how React refs let us delegate DOM manipulation for our data visualizations, and how Observable’s embedding functionality lets us easily repurpose community-built visualizations for our own data
&
use cases. By the end of this talk we’ll know how to get a beautiful, customized, interactive data visualization into our apps with a fraction of the time
&
effort!


Workshops on related topic

React Advanced Conference 2021React Advanced Conference 2021
132 min
Concurrent Rendering Adventures in React 18
Workshop Free
With the release of React 18 we finally get the long awaited concurrent rendering. But how is that going to affect your application? What are the benefits of concurrent rendering in React? What do you need to do to switch to concurrent rendering when you upgrade to React 18? And what if you don’t want or can’t use concurrent rendering yet?
There are some behavior changes you need to be aware of! In this workshop we will cover all of those subjects and more.
Join me with your laptop in this interactive workshop. You will see how easy it is to switch to concurrent rendering in your React application. You will learn all about concurrent rendering, SuspenseList, the startTransition API and more.
React Summit Remote Edition 2021React Summit Remote Edition 2021
177 min
React Hooks Tips Only the Pros Know
Workshop
The addition of the hooks API to React was quite a major change. Before hooks most components had to be class based. Now, with hooks, these are often much simpler functional components. Hooks can be really simple to use. Almost deceptively simple. Because there are still plenty of ways you can mess up with hooks. And it often turns out there are many ways where you can improve your components a better understanding of how each React hook can be used.
You will learn all about the pros and cons of the various hooks. You will learn when to use useState() versus useReducer(). We will look at using useContext() efficiently. You will see when to use useLayoutEffect() and when useEffect() is better.


React Advanced Conference 2021React Advanced Conference 2021
174 min
React, TypeScript, and TDD
Workshop Free
ReactJS is wildly popular and thus wildly supported. TypeScript is increasingly popular, and thus increasingly supported.
The two together? Not as much. Given that they both change quickly, it's hard to find accurate learning materials.
React+TypeScript, with JetBrains IDEs? That three-part combination is the topic of this series. We'll show a little about a lot. Meaning, the key steps to getting productive, in the IDE, for React projects using TypeScript. Along the way we'll show test-driven development and emphasize tips-and-tricks in the IDE.


React Summit 2023React Summit 2023
171 min
React Performance Debugging Masterclass
Workshop Free
Ivan’s first attempts at performance debugging were chaotic. He would see a slow interaction, try a random optimization, see that it didn't help, and keep trying other optimizations until he found the right one (or gave up).
Back then, Ivan didn’t know how to use performance devtools well. He would do a recording in Chrome DevTools or React Profiler, poke around it, try clicking random things, and then close it in frustration a few minutes later. Now, Ivan knows exactly where and what to look for. And in this workshop, Ivan will teach you that too.
Here’s how this is going to work. We’ll take a slow app → debug it (using tools like Chrome DevTools, React Profiler, and why-did-you-render) → pinpoint the bottleneck → and then repeat, several times more. We won’t talk about the solutions (in 90% of the cases, it’s just the ol’ regular useMemo() or memo()). But we’ll talk about everything that comes before – and learn how to analyze any React performance problem, step by step.
(Note: This workshop is best suited for engineers who are already familiar with how useMemo() and memo() work – but want to get better at using the performance tools around React. Also, we’ll be covering interaction performance, not load speed, so you won’t hear a word about Lighthouse 🤐)
React Advanced Conference 2021React Advanced Conference 2021
145 min
Web3 Workshop - Building Your First Dapp
Workshop Free
In this workshop, you'll learn how to build your first full stack dapp on the Ethereum blockchain, reading and writing data to the network, and connecting a front end application to the contract you've deployed. By the end of the workshop, you'll understand how to set up a full stack development environment, run a local node, and interact with any smart contract using React, HardHat, and Ethers.js.


React Summit 2023React Summit 2023
152 min
Designing Effective Tests With React Testing Library
Workshop
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents
- The different kinds of React application tests, and where component tests fit in
- A mental model for thinking about the inputs and outputs of the components you test
- Options for selecting DOM elements to verify and interact with them
- The value of mocks and why they shouldn’t be avoided
- The challenges with asynchrony in RTL tests and how to handle them
Prerequisites
- Familiarity with building applications with React
- Basic experience writing automated tests with Jest or another unit testing framework
- You do not need any experience with React Testing Library
- Machine setup: Node LTS, Yarn