Build a UI that Learns - Intelligent Prefetching with React and TensorFlow.js

Rate this content
Bookmark

How to build a UI that LEARNS? Being able to learn and predict the behavior of users has many powerful applications, one of them is the chance to boost the UI performance prefetching code & resources before the user reaches them. In this talk, we describe a high-level implementation of an intelligent prefetcher, using ReactJS and TensorFlow.js. We use neural networks to learn the user's behaviour, and leverages React's lazy-loading API to prefetch components according to predictions. There is a chance for Frontend developers to explore the powerful combination of UI and AI.

17 min
14 May, 2021

Video Summary and Transcription

Today's talk explores intelligent prefetching in React, including code splitting, lazy loading, and prefetching to improve performance. The use of neural networks for sequence prediction and training with actual user behavior is discussed. React context is used to link UI handlers with predictions and prefetching, enabling dynamic content import and improved user experience. The combination of AI and UI development is showcased in this personal project.

Available in Español

1. Introduction to Intelligent Prefetching in React

Short description:

Today's talk is about intelligent prefetching in React. We discuss the issues with bundle size and loading times in single-page applications and how React code splitting can help. We also explore the concept of prefetching components and using lazy loading for entire routes to improve performance. Additionally, we delve into the question of predicting user behavior and how supervised learning, specifically neural networks, can be used for sequence prediction.

Hello all and welcome to this session. My name is Eli Ranatan, and I'm happy to be here in React Summit. Today I will talk about intelligent prefetching in React. So we start by talking about prefetching of code, later we show how we can predict the user behavior using machine learning, and finally we combine those solutions to form fast React applications.

So I want to start by talking about the context. Usually we are building those amazing single-page applications and eventually we turn up with this huge bundle size that eventually causes problems in loading times, slowness, and potentially harming the user experience. So what we can do is to use the React code splitting API in order to import components on demand. So here instead of just importing the chart component, increasing my bundle size, I'm lazy loading the chart component and the actual fetching of code would happen only when we render it. But that does not solve the problem entirely, right? Because we are just shifting the fetching time to somewhere else. Whenever the user will actually want to render this chart component, then it will have to wait for the fetching to occur, and that can affect the loading time and harm the user experience. So what about this crazy idea? We could prefetch, we could break this trade-off by prefetching the chart component. So using the time that the user is just staring at the screen and before it ever reaches the we could prepare this content, we could use the dynamic import feature to dynamically import the component and then overriding this variable. Now, we could do that if we had some indication or some educated guess about the user's next move. We could use this trick to further increase the performance of our applications if we are doing that for entire routes. So if I have this routing between dashboards and products, I could use a lazy loading to lazy load those routes. And then conditionally, if I have some indication about the user's next move, I could dynamically import those routes, saving a lot of time.

So that brings us to the question, and that's a very interesting question, of how we can predict the next move of user. So let's just analyze this question for a bit. We have this complicated single-page application that is combined from a lot of different components. And we can list those different triggers, those actions, links, buttons that the user can interact with, or at least those that are interesting in the sense that they would cause a rendering of other large components. Now we could keep track on the user's behavior and make this perform this ordered sequence of interactions that the user is performing. And the question is how we can base on this sequence how we can predict the next element in that sequence. So what we need is this intelligent mechanism that receives a sequence and returns the prediction or estimation of the next item in that sequence. And in machine learning we call this problem a sequence prediction. So we can use supervised learning specifically neural network in order to figure out this estimation. So a neural network will receive an encoding of that sequence as this series of numbers. Each number is an identification of a certain trigger and returns this probability distribution across all the possible triggers that the user can interact with. So each output would represent the chance that that corresponding element will be the next element in the sequence. So this is a supervised learning and the neural network acts as a function approximator. The function receives a sequence and returns a probability distribution.

2. Training Neural Network with User Behavior

Short description:

We use the user's actual behavior as examples to train the neural network. One-hot encoding is used to break numerical correlations between elements. The LSTM type of neural network is crucial for sequence predictions. TensorFlow.js library enables implementation in the browser environment. The dot predict command provides probability predictions for each element. Training the network with the dot fit command using actual user behavior as examples can be time-consuming.

Now once that we have this output, we could just take the maximal argument and derive from that the next UI element that the user is about to interact with. Now of course that those predictions will be meaningless unless we train the network. So in supervised learning, we have to supply the network with examples. And the best source for those examples is the actual behavior of the user. So we could sample the behavior of the user, taking those sequences and feed the neural network with those examples, basically telling the neural network this is the actual behavior of the user, please adjust your predictions accordingly.

Now I think it's cool that we are using the user in order to train the application. So the more that the user uses the application, then it trains it accordingly. And when the user changes its behavior over time, then the application can adapt. Now specifically, speaking about implementation, this predictor can be implemented as a neural network. And what you are seeing here is that the input of that neural network will be... We are using here one-hot encoding instead of just pure numbers. And this is because we want to break any numerical correlation between those elements. There is no meaning to say that a dashboard is smaller than products, because it is represented by the number three and not by the number 21. So this is the way to break this correlation.

Now once that we have... The most important thing in the architecture of your neural network is that we have to use this LSTM type of neural network. This is the best type that can support sequence predictions. It's also important to notice that the input layer is here. The number of units in the input layer should match the length of the sequences that we are working with, and the number of units in the output there should match the number... All the possible triggers that the user can interact with. Now, once that we have the output from the network, we can attach it to the corresponding element in the user experience and assume that the user is about to hit this element.

Now, implementation-wise, we have to implement this mechanism somewhere in the browser environment, and we can do that using TensorFlow.js library. So TensorFlow.js library is based on WebGL, and it allows us to implement machine learning algorithms within the browser environment. So we are using the sequential command in order to stack layers in the network one after the other, and we are specifying these LSTM type layer, specifying all the shape of the input as a matrix of binaric numbers. And eventually, once we have this network, we can use the dot predict command supplying a sequence and asking to predict the probability for each element. Now, those predictions will be meaningless unless we train the network, as we say, and for the training part, we would use the dot fit command. So we're supplying this batch of sequences and the correlated labels. And those labels represent elements. So we are taking the actual behavior of the user and supply it as examples to the fitting mechanism. Now, training the network could take time.

3. Linking Predictions and Prefetching in React

Short description:

To link predictions with prefetching, we wrap the show chart button with a custom component called prediction link. This component connects to a React context that encapsulates the prediction logic. The onPrediction handler is triggered when a prediction matches the prediction link. Another React context acts as a message bus for prefetching, allowing consumers to dynamically import necessary content. The entire app can access the predictor through a higher level provider.

That's a heavy operation. And we don't want to disturb the user experience. So we're using the fact that the fitting command is asynchronous. And we can basically do it in the background without disturbing the user.

Now, the next challenge that we need to solve here is how we should link between those predictions that we are doing and the prefetching that we discussed earlier. So what we want to do is to prefetch content whenever we know that the user is about to hit a certain button. Basically, making the application faster and faster. And the more that the user uses the app, the faster it gets.

So we have this button somewhere in the application that is called show chart. And we can wrap it with this custom component that I call prediction link. And this prediction link will be connected to a React context. The purpose of React context is to supply this global state that is available for the entire app. And here I'm using a React context to encapsulate all the prediction logic. So it would hold all the implementation of the neural network together with the TensorFlow code and so on. And it will be available through the entire app. So those prediction components can query the predictor for the current prediction with the next move of the user. And they can expose this onPrediction handler that would be triggered upon an associated prediction.

Now this onPrediction, we could use this trigger to communicate with another React context, which is responsible for the prefetching. So it basically acts as a message bus announcing that the content that is associated with this button should be prefetched. Now any consumer that is interested in this information can use it and use Dynamic Imports to dynamically import the necessary content. So what we're doing here is connecting between predictions and prefetching using two React contexts.

Now let's see how that works code-wise. So we have this button somewhere in the application, this show chart. We can wrap it with a prediction link custom component. Now this prediction link exposes the on prediction trigger. Whenever there is a prediction for that key chart, then we could do something with this prediction. Let's take a closer look on how this prediction link is implemented. We are using the use context hook to connect to the predictor context, and then we are querying this predictor context for the current prediction. If the current prediction matches this specific prediction link, then we are triggering the on prediction handler. Now, in order to have the predictor available through the entire app, we have to wrap the with this higher level provider.

4. Leveraging AI and UI Integration

Short description:

Now that we have the prediction link, we can prefetch the necessary content associated with the button. We can also dynamically import routes based on the prefetcher context. We used React code splitting, lazy loading, and prefetching to improve performance. Additionally, we demonstrated predicting user behavior using a machine learning-based click predictor in the browser environment with TensorFlow.js. React context was used to link UI handlers and predictions, and another context linked predictions and prefetching. This personal project showcases the combination of AI and UI using machine learning for UI development.

Now that we have this prediction link, we can use it to call the prefetch component context using this trigger. Basically saying, please prefetch the necessary content that is associated with this button.

Now going back to the component that actually fetched code and rendered the necessary content, we could again use the use context hook to connect to the prefetcher context, query that context, and basically asking if the component that we should prefetch is the chart component. And if so, we are using the dynamic imports to do that.

And of course, we can do the same thing with entire routes. So if we have this routing between dashboard and products, we could dynamically import those routes conditionally, based on the prefetcher context.

So I want to summarize everything that we have discussed here. The first thing that we did was to use the React code splitting API in order to improve the performance of our application. Later, we have combined the lazy loading and prefetching of content in order to further improve the performance of our application, basically using the time that the user is staring at the screen in order to prepare the next content.

Then we showed how we can predict the user behavior using a machine learning based click predictor in the browser environment using TensorFlow.js. Of course, that predicting the user behavior have more applications that are very interesting to think of.

Now we showed that we can use React context to link between UI handlers and predictions. And finally, we could use another React context to link between predictions and prefetch. So that's the entire overview. That's a personal project that I thought of. And I hope that will bring you some inspiration about how we can combine AI and UI and basically using machine learning in order to leverage UI development. Thank you.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

React Advanced Conference 2022React Advanced Conference 2022
25 min
A Guide to React Rendering Behavior
Top Content
React is a library for "rendering" UI from components, but many users find themselves confused about how React rendering actually works. What do terms like "rendering", "reconciliation", "Fibers", and "committing" actually mean? When do renders happen? How does Context affect rendering, and how do libraries like Redux cause updates? In this talk, we'll clear up the confusion and provide a solid foundation for understanding when, why, and how React renders. We'll look at: - What "rendering" actually is - How React queues renders and the standard rendering behavior - How keys and component types are used in rendering - Techniques for optimizing render performance - How context usage affects rendering behavior| - How external libraries tie into React rendering
React Summit Remote Edition 2021React Summit Remote Edition 2021
33 min
Building Better Websites with Remix
Top Content
Remix is a new web framework from the creators of React Router that helps you build better, faster websites through a solid understanding of web fundamentals. Remix takes care of the heavy lifting like server rendering, code splitting, prefetching, and navigation and leaves you with the fun part: building something awesome!
React Advanced Conference 2023React Advanced Conference 2023
33 min
React Compiler - Understanding Idiomatic React (React Forget)
React provides a contract to developers- uphold certain rules, and React can efficiently and correctly update the UI. In this talk we'll explore these rules in depth, understanding the reasoning behind them and how they unlock new directions such as automatic memoization. 
React Advanced Conference 2022React Advanced Conference 2022
30 min
Using useEffect Effectively
Top Content
Can useEffect affect your codebase negatively? From fetching data to fighting with imperative APIs, side effects are one of the biggest sources of frustration in web app development. And let’s be honest, putting everything in useEffect hooks doesn’t help much. In this talk, we'll demystify the useEffect hook and get a better understanding of when (and when not) to use it, as well as discover how declarative effects can make effect management more maintainable in even the most complex React apps.
React Summit 2022React Summit 2022
20 min
Routing in React 18 and Beyond
Top Content
Concurrent React and Server Components are changing the way we think about routing, rendering, and fetching in web applications. Next.js recently shared part of its vision to help developers adopt these new React features and take advantage of the benefits they unlock.In this talk, we’ll explore the past, present and future of routing in front-end applications and discuss how new features in React and Next.js can help us architect more performant and feature-rich applications.
React Advanced Conference 2021React Advanced Conference 2021
27 min
(Easier) Interactive Data Visualization in React
Top Content
If you’re building a dashboard, analytics platform, or any web app where you need to give your users insight into their data, you need beautiful, custom, interactive data visualizations in your React app. But building visualizations hand with a low-level library like D3 can be a huge headache, involving lots of wheel-reinventing. In this talk, we’ll see how data viz development can get so much easier thanks to tools like Plot, a high-level dataviz library for quick & easy charting, and Observable, a reactive dataviz prototyping environment, both from the creator of D3. Through live coding examples we’ll explore how React refs let us delegate DOM manipulation for our data visualizations, and how Observable’s embedding functionality lets us easily repurpose community-built visualizations for our own data & use cases. By the end of this talk we’ll know how to get a beautiful, customized, interactive data visualization into our apps with a fraction of the time & effort!

Workshops on related topic

React Summit 2023React Summit 2023
170 min
React Performance Debugging Masterclass
Featured WorkshopFree
Ivan’s first attempts at performance debugging were chaotic. He would see a slow interaction, try a random optimization, see that it didn't help, and keep trying other optimizations until he found the right one (or gave up).
Back then, Ivan didn’t know how to use performance devtools well. He would do a recording in Chrome DevTools or React Profiler, poke around it, try clicking random things, and then close it in frustration a few minutes later. Now, Ivan knows exactly where and what to look for. And in this workshop, Ivan will teach you that too.
Here’s how this is going to work. We’ll take a slow app → debug it (using tools like Chrome DevTools, React Profiler, and why-did-you-render) → pinpoint the bottleneck → and then repeat, several times more. We won’t talk about the solutions (in 90% of the cases, it’s just the ol’ regular useMemo() or memo()). But we’ll talk about everything that comes before – and learn how to analyze any React performance problem, step by step.
(Note: This workshop is best suited for engineers who are already familiar with how useMemo() and memo() work – but want to get better at using the performance tools around React. Also, we’ll be covering interaction performance, not load speed, so you won’t hear a word about Lighthouse 🤐)
React Advanced Conference 2021React Advanced Conference 2021
132 min
Concurrent Rendering Adventures in React 18
Top Content
Featured WorkshopFree
With the release of React 18 we finally get the long awaited concurrent rendering. But how is that going to affect your application? What are the benefits of concurrent rendering in React? What do you need to do to switch to concurrent rendering when you upgrade to React 18? And what if you don’t want or can’t use concurrent rendering yet?

There are some behavior changes you need to be aware of! In this workshop we will cover all of those subjects and more.

Join me with your laptop in this interactive workshop. You will see how easy it is to switch to concurrent rendering in your React application. You will learn all about concurrent rendering, SuspenseList, the startTransition API and more.
React Summit Remote Edition 2021React Summit Remote Edition 2021
177 min
React Hooks Tips Only the Pros Know
Top Content
Featured Workshop
The addition of the hooks API to React was quite a major change. Before hooks most components had to be class based. Now, with hooks, these are often much simpler functional components. Hooks can be really simple to use. Almost deceptively simple. Because there are still plenty of ways you can mess up with hooks. And it often turns out there are many ways where you can improve your components a better understanding of how each React hook can be used.You will learn all about the pros and cons of the various hooks. You will learn when to use useState() versus useReducer(). We will look at using useContext() efficiently. You will see when to use useLayoutEffect() and when useEffect() is better.
React Advanced Conference 2021React Advanced Conference 2021
174 min
React, TypeScript, and TDD
Top Content
Featured WorkshopFree
ReactJS is wildly popular and thus wildly supported. TypeScript is increasingly popular, and thus increasingly supported.

The two together? Not as much. Given that they both change quickly, it's hard to find accurate learning materials.

React+TypeScript, with JetBrains IDEs? That three-part combination is the topic of this series. We'll show a little about a lot. Meaning, the key steps to getting productive, in the IDE, for React projects using TypeScript. Along the way we'll show test-driven development and emphasize tips-and-tricks in the IDE.
React Advanced Conference 2021React Advanced Conference 2021
145 min
Web3 Workshop - Building Your First Dapp
Top Content
Featured WorkshopFree
In this workshop, you'll learn how to build your first full stack dapp on the Ethereum blockchain, reading and writing data to the network, and connecting a front end application to the contract you've deployed. By the end of the workshop, you'll understand how to set up a full stack development environment, run a local node, and interact with any smart contract using React, HardHat, and Ethers.js.
React Summit 2023React Summit 2023
151 min
Designing Effective Tests With React Testing Library
Featured Workshop
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents- The different kinds of React application tests, and where component tests fit in- A mental model for thinking about the inputs and outputs of the components you test- Options for selecting DOM elements to verify and interact with them- The value of mocks and why they shouldn’t be avoided- The challenges with asynchrony in RTL tests and how to handle them
Prerequisites- Familiarity with building applications with React- Basic experience writing automated tests with Jest or another unit testing framework- You do not need any experience with React Testing Library- Machine setup: Node LTS, Yarn