Scale Your React App without Micro-frontends

Rate this content
Bookmark

As your team grows and becomes multiple teams, the size of your codebase follows. You get to 100k lines of code and your build time dangerously approaches the 10min mark 😱 But that’s not all, your static CI checks (linting, type coverage, dead code) and tests are also taking longer and longer...

How do you keep your teams moving fast and shipping features to users regularly if your PRs take forever to be tested and deployed?

After exploring a few options we decided to go down the Nx route. Let’s look at how to migrate a large codebase to Nx and take advantage of its incremental builds!

Jonathan Wagner
Jonathan Wagner
21 min
21 Jun, 2022

Comments

Sign in or register to post your comment.

Video Summary and Transcription

This Talk discusses scaling a React app without micro-frontend and the challenges of a growing codebase. Annex is introduced as a tool for smart rebuilds and computation caching. The importance of libraries in organizing code and promoting clean architecture is emphasized. The use of caching, NxCloud, and incremental build for optimization is explored. Updating dependencies and utilizing profiling tools are suggested for further performance improvements. Splitting the app into libraries and the benefits of a build system like NX are highlighted.

Available in Español

1. Scaling React App Without Micro-Frontend

Short description:

Welcome to my talk on scaling your React app without micro-frontend. I'll discuss the problem of scaling up a codebase, my journey to fixing it, and what I've learned along the way. When a codebase grows, the CI becomes slower, causing a slower feedback loop for developers and unhappy users. In my case, the build time reached over 30 minutes, which was unacceptable. I tried over-engineering the CI, but it became difficult to manage. Then, I discovered Annex and learned how to use it properly to only rerun things that have changed.

Hi everyone, welcome to my talk on scaling your React app without micro-frontend. Very quick disclaimer first, this talk is about scalability of your codebase on a developer perspective and not on user-facing performances. That's it for the disclaimer.

Hello again, I'm Jonathan Wagner I'm an engineering manager at DataUK and I've been working for the past 4 years on a bit more than 10 projects in production and for the last 8 months I've been working with NX, which is a build system helping you build faster and do things faster. That's going to be the core of the talk here. Let's have a look at what we'll talk about.

First of all, discuss a bit of the problem, what happens when you scale up codebase and then my journey to fixing it and what I've learned along the way, what I've tried, what didn't work, what worked. So first of all, the problem. I mentioned, having a codebase that grows, that means your CI becomes a bit slower and when it becomes slower, it means you have a slower feedback loop, which means your developers are unhappy, it takes longer to develop features and in the end your users are unhappy, which we definitely don't want.

So when your codebase grows, you have type checking, you have ESLint, you have maybe some dead code tests, some unit tests, end-to-end tests, a lot of testing and the build time and then everything's taking a bit of time and it adds up and in my case it added up to more than 30 minutes and that's my trigger. Ideally I want my site to be 10 or 15 minutes, when it reaches 30 minutes it means something's gone terribly wrong and I want to address it. In this case, we cannot even see the buildable time, it's just skyrocketing, it's going to 3-4 hours, which means it's costing a lot of money to the company and just frustrating for everyone.

Let's try to look at a more precise example here, so we have a growing codebase which means we may have maybe 800 tests. Imagine you have a pull request where I do one line change, you push everything and the CI runs everything again, so that means you have to wait 20 minutes, maybe 30 minutes for the test to pass. Does it sound normal to you that a one line change triggers 20 minutes running the CI? I don't think so. And nx doesn't think so either. From the nx docs they actually say, the duration of the invoked operation should be proportional to the size of the change, and something very strong. It seems simple but it's quite tricky to put in place. It involves a bit of caching, a lot of caching and then putting all that together properly. But I didn't know that at first.

So obviously I tried to just over-engine my CI and that meant doing a lot of parallelisation, putting some tests up to skip frontend or backend tests, depending on what was changed. And that meant writing a lot of custom rules, which were tricky and introduced a few regressions. And even like switching TypeScript compilations to something based on Rust like swc. And doing all of this, I got some improvements, got down to 20 minutes each but it meant I had a tricky CI to manage. We had 27 different jobs and it meant understanding how they all played together, which ones were conditional on which, and it became just tricky to manage and to maintain. So that's the start of where I wanted to go afterwards. I think we spent hours and hours optimising the CI. Where do we go from there? How do we make it faster without overengineering it more? Here comes the journey. Discovering Annex, discovering a bit more about how to use Annex properly, what the secret trick is, and how the caching puts everything and stitching everything together. As I said before, the main idea we want to aim for is only rerunning things that have changed.

2. Annex and the Concept of Libraries

Short description:

Annex does smart rebuilds, only on the things that have changed and have been affected. It uses computation caching and helps you generate things so you don't have to do them every time. The secret trick is to use libraries everywhere. A library is simply a folder where you can execute operations specific to the code it concerns. Libraries help organize the code in the folder structure and can be easily generated. When a change occurs, Annex only tests and relints the affected projects, following the core principle of testing only what has changed. Libraries also promote clean architecture by forcing developers to consider where to put their code and prevent spaghetti code.

That's basically what Annex does. It does smart rebuilds, only on the things that have changed and that have been affected. For that, it uses computation caching, and it helps you generate things so that you don't have to do them hand in hand every time. Basically, Annex was already set up on the project. We weren't using it for the build system. We were just using it for the monoreport management. And when I realized how much it could do, that was the open door to so much more.

So, now that we know Annex can do all of this, how does it actually work? I've mentioned a secret trick. It's actually not so secret. It's advertised everywhere on the Annex documentation. The idea is to use libraries everywhere. You might wonder, what is a library in Annex? It's simply a folder where you can execute a few operations, like testing, linting, basically anything you want. It's just about the code that it concerns. So, if we look at, for example, our project, we had here the front-end, which was the app that was actually deployed and used by our users. And then everything else here below are libraries. So, the app is using libraries. And then it gets deployed and the libraries are there for a way of organizing the code in your folder structure. And the thing is, you can generate them easily so that it's just a common way to create a new one and doesn't cost you much. And what we can see here as well is that, in pink, it's highlighting the projects that have been affected by change. So, in this case, I made a change in the design system and it shows me that a few libraries have been affected and the app as well. So, when I run my test or when I run the lint, it's only going to test and relint those pink projects, so the libraries and the app. It's not going to touch anything else, because it hasn't changed. So, it doesn't need to test it, and that's the beauty of it. That's the core principle.

The concept of libraries might still be a bit blurry. So, let's have a look here at what it means concretely. So, this is an example repository from NX, where you have basically a couple apps, cards, products, and then a few libraries. And if we zoom in on the libraries, it's actually just a nested config, a jest config, and then a TypeScript config, and then all your source code in the src folder, and it's as simple as that. There's not much more, and it's everything you need to be able to run your operations in each of those folders. The side bonus that you have from using libraries is that it forces you to have a clean architecture, because you have to ask yourself the question, where should I put my code, where should I put it into the same library, should I create a new library? It's a question that because you ask yourself, it forces you to trigger a discussion, think about it, put it in the right place, and it saves you some time and prevents spaghetti code from happening in the future.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Don't Solve Problems, Eliminate Them
React Advanced Conference 2021React Advanced Conference 2021
39 min
Don't Solve Problems, Eliminate Them
Top Content
Humans are natural problem solvers and we're good enough at it that we've survived over the centuries and become the dominant species of the planet. Because we're so good at it, we sometimes become problem seekers too–looking for problems we can solve. Those who most successfully accomplish their goals are the problem eliminators. Let's talk about the distinction between solving and eliminating problems with examples from inside and outside the coding world.
Using useEffect Effectively
React Advanced Conference 2022React Advanced Conference 2022
30 min
Using useEffect Effectively
Top Content
Can useEffect affect your codebase negatively? From fetching data to fighting with imperative APIs, side effects are one of the biggest sources of frustration in web app development. And let’s be honest, putting everything in useEffect hooks doesn’t help much. In this talk, we'll demystify the useEffect hook and get a better understanding of when (and when not) to use it, as well as discover how declarative effects can make effect management more maintainable in even the most complex React apps.
Design Systems: Walking the Line Between Flexibility and Consistency
React Advanced Conference 2021React Advanced Conference 2021
47 min
Design Systems: Walking the Line Between Flexibility and Consistency
Top Content
Design systems aim to bring consistency to a brand's design and make the UI development productive. Component libraries with well-thought API can make this a breeze. But, sometimes an API choice can accidentally overstep and slow the team down! There's a balance there... somewhere. Let's explore some of the problems and possible creative solutions.
React Concurrency, Explained
React Summit 2023React Summit 2023
23 min
React Concurrency, Explained
Top Content
React 18! Concurrent features! You might’ve already tried the new APIs like useTransition, or you might’ve just heard of them. But do you know how React 18 achieves the performance wins it brings with itself? In this talk, let’s peek under the hood of React 18’s performance features: - How React 18 lowers the time your page stays frozen (aka TBT) - What exactly happens in the main thread when you run useTransition() - What’s the catch with the improvements (there’s no free cake!), and why Vue.js and Preact straight refused to ship anything similar
Levelling up Monorepos with npm Workspaces
DevOps.js Conf 2022DevOps.js Conf 2022
33 min
Levelling up Monorepos with npm Workspaces
Top Content
Learn more about how to leverage the default features of npm workspaces to help you manage your monorepo project while also checking out some of the new npm cli features.
TypeScript and React: Secrets of a Happy Marriage
React Advanced Conference 2022React Advanced Conference 2022
21 min
TypeScript and React: Secrets of a Happy Marriage
Top Content
TypeScript and React are inseparable. What's the secret to their successful union? Quite a lot of surprisingly strange code. Learn why useRef always feels weird, how to wrangle generics in custom hooks, and how union types can transform your components.

Workshops on related topic

React Performance Debugging Masterclass
React Summit 2023React Summit 2023
170 min
React Performance Debugging Masterclass
Top Content
Featured WorkshopFree
Ivan Akulov
Ivan Akulov
Ivan’s first attempts at performance debugging were chaotic. He would see a slow interaction, try a random optimization, see that it didn't help, and keep trying other optimizations until he found the right one (or gave up).
Back then, Ivan didn’t know how to use performance devtools well. He would do a recording in Chrome DevTools or React Profiler, poke around it, try clicking random things, and then close it in frustration a few minutes later. Now, Ivan knows exactly where and what to look for. And in this workshop, Ivan will teach you that too.
Here’s how this is going to work. We’ll take a slow app → debug it (using tools like Chrome DevTools, React Profiler, and why-did-you-render) → pinpoint the bottleneck → and then repeat, several times more. We won’t talk about the solutions (in 90% of the cases, it’s just the ol’ regular useMemo() or memo()). But we’ll talk about everything that comes before – and learn how to analyze any React performance problem, step by step.
(Note: This workshop is best suited for engineers who are already familiar with how useMemo() and memo() work – but want to get better at using the performance tools around React. Also, we’ll be covering interaction performance, not load speed, so you won’t hear a word about Lighthouse 🤐)
React Hooks Tips Only the Pros Know
React Summit Remote Edition 2021React Summit Remote Edition 2021
177 min
React Hooks Tips Only the Pros Know
Top Content
Featured Workshop
Maurice de Beijer
Maurice de Beijer
The addition of the hooks API to React was quite a major change. Before hooks most components had to be class based. Now, with hooks, these are often much simpler functional components. Hooks can be really simple to use. Almost deceptively simple. Because there are still plenty of ways you can mess up with hooks. And it often turns out there are many ways where you can improve your components a better understanding of how each React hook can be used.You will learn all about the pros and cons of the various hooks. You will learn when to use useState() versus useReducer(). We will look at using useContext() efficiently. You will see when to use useLayoutEffect() and when useEffect() is better.
React, TypeScript, and TDD
React Advanced Conference 2021React Advanced Conference 2021
174 min
React, TypeScript, and TDD
Top Content
Featured WorkshopFree
Paul Everitt
Paul Everitt
ReactJS is wildly popular and thus wildly supported. TypeScript is increasingly popular, and thus increasingly supported.

The two together? Not as much. Given that they both change quickly, it's hard to find accurate learning materials.

React+TypeScript, with JetBrains IDEs? That three-part combination is the topic of this series. We'll show a little about a lot. Meaning, the key steps to getting productive, in the IDE, for React projects using TypeScript. Along the way we'll show test-driven development and emphasize tips-and-tricks in the IDE.
Web3 Workshop - Building Your First Dapp
React Advanced Conference 2021React Advanced Conference 2021
145 min
Web3 Workshop - Building Your First Dapp
Top Content
Featured WorkshopFree
Nader Dabit
Nader Dabit
In this workshop, you'll learn how to build your first full stack dapp on the Ethereum blockchain, reading and writing data to the network, and connecting a front end application to the contract you've deployed. By the end of the workshop, you'll understand how to set up a full stack development environment, run a local node, and interact with any smart contract using React, HardHat, and Ethers.js.
Designing Effective Tests With React Testing Library
React Summit 2023React Summit 2023
151 min
Designing Effective Tests With React Testing Library
Top Content
Featured Workshop
Josh Justice
Josh Justice
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents- The different kinds of React application tests, and where component tests fit in- A mental model for thinking about the inputs and outputs of the components you test- Options for selecting DOM elements to verify and interact with them- The value of mocks and why they shouldn’t be avoided- The challenges with asynchrony in RTL tests and how to handle them
Prerequisites- Familiarity with building applications with React- Basic experience writing automated tests with Jest or another unit testing framework- You do not need any experience with React Testing Library- Machine setup: Node LTS, Yarn
Remix Fundamentals
React Summit 2022React Summit 2022
136 min
Remix Fundamentals
Top Content
Featured WorkshopFree
Kent C. Dodds
Kent C. Dodds
Building modern web applications is riddled with complexity And that's only if you bother to deal with the problems
Tired of wiring up onSubmit to backend APIs and making sure your client-side cache stays up-to-date? Wouldn't it be cool to be able to use the global nature of CSS to your benefit, rather than find tools or conventions to avoid or work around it? And how would you like nested layouts with intelligent and performance optimized data management that just works™?
Remix solves some of these problems, and completely eliminates the rest. You don't even have to think about server cache management or global CSS namespace clashes. It's not that Remix has APIs to avoid these problems, they simply don't exist when you're using Remix. Oh, and you don't need that huge complex graphql client when you're using Remix. They've got you covered. Ready to build faster apps faster?
At the end of this workshop, you'll know how to:- Create Remix Routes- Style Remix applications- Load data in Remix loaders- Mutate data with forms and actions