Build a UI that Learns - Intelligent Prefetching with React and TensorFlow.js

Hello all and welcome to the session. My name is Eliran Atan and I'm happy to be here in react Summit. Today I will talk about intelligent prefetching in react. So we start by talking about prefetching of code, later we show how we can predict the user behavior using machine learning, and finally we combine those solutions to form fast react applications. So I want to start by talking about the context. Usually we are building those amazing single page applications and eventually we turn up with this huge bundle size that eventually causes problems in loading times, slowness, and potentially harming the user experience. So what we can do is to use the react code splitting api in order to import components on demand. So here instead of just importing the chart component, increasing my bundle size, I'm lazy loading the chart component and the actual fetching of code would happen only when we render it. But that does not solve the problem entirely, right? Because we are just shifting the fetching time to somewhere else. Whenever the user would actually want to render this chart component, then it will have to wait for the fetching to occur and that can affect the loading time and harm the user experience. So what about this crazy idea? We could prefetch, we could break this tradeoff by prefetching the chart component. So using the time that the user is just staring at the screen and before it ever reaches the chart, we could prepare this content. We could use the dynamic import feature to dynamically import the component and then overriding this variable. Now we could do that if we had some indication or some educated guess about the user's next move. We could use this trick to further increase the performance of our applications if we are doing that for entire routes. So if I have this routing between dashboards and products, I could use a lazy loading to lazy load those routes. And then conditionally, if I have some indication about the user's next move, I could dynamically import those routes, saving a lot of time. So that brings us to the question, and that's a very interesting question, of how we can predict the next move of user. So let's just analyze this question for a bit. We have this complicated single page application that is combined from a lot of different components. And we can list those different triggers, those actions, links, buttons that the user can interact with, or at least those that are interesting in the sense that they would cause a rendering of other large components. Now we could keep track on the user's behavior and make this, perform this ordered sequence of interactions that the user is performing. And the question is how we can base, based on this sequence, how we can predict the next element in that sequence. So what we need is this intelligent mechanism that receives a sequence and returns the prediction or estimation of the next item in that sequence. And in machine learning, we call this problem a sequence prediction. So we can use supervised learning, specifically neural network, in order to figure out this estimation. So a neural network will receive an encoding of that sequence as this series of numbers. Each number is the identification of a certain trigger and returns this probability distribution across all the possible triggers that the user can interact with. So each output would represent the chance that that corresponding element will be the next element in the sequence. So this is a supervised learning and the neural network acts as a function approximator. The function that receives a sequence and returns a probability distribution. Now once that we have this output, we could just take the maximal argument and derive from that the next UI element that the user is about to interact with. Now of course those predictions will be meaningless unless we train the network. So in supervised learning, we have to supply the network with examples. And the best source for those examples is the actual behavior of the user. So we could sample the behavior of the user, taking those sequences and feed the neural network with those examples. Basically telling the neural network, this is the actual behavior of the user, please adjust your predictions accordingly. So I think it's cool that we are using the user in order to train the application. So the more that the user uses the application, then it trains it accordingly. And when the user changes its behavior over time, then the application can adapt. Now specifically speaking about implementation, this predictor can be implemented as a neural network. And what we are seeing here is that the input of that neural network will be, we are using here one-hot encoding instead of just pure numbers. And this is because we want to break any numerical correlation between those elements. There is no meaning to say that a dashboard is smaller than products because it is represented by the number three and not by the number 21. So this is the way to break this correlation. Now once that we have, the most important thing in the architecture of your neural network is that we have to use this LSTM type of neural network. This is the best type that can support sequence predictions. It is also important to notice that the input layer is here, the number of units in the input layer should match the length of the sequences that we are working with. And the number of units in the output layer should match the number, all the possible triggers that the user can interact with. Now once that we have the output from that network, we can attach it to the corresponding element in the user experience and assume that the user is about to hit this element. Now implementation wise, we have to implement this mechanism somewhere in the browser environment. And we can do that using tensorflow.js library. So tensorflow.js library is based on webgl and it allows us to implement machine learning algorithms within the browser environment. So we are using the sequential command in order to stack layers in the network one after the another, and we are specifying this LSTM type layer, specifying all the shape of the input as a matrix of binaric numbers. And eventually, once we have this network, we can use the dot predict command supplying a sequence and asking to predict the probability for each element. Now those predictions will be meaningless unless we train the network, as we said. And for the training part, we would use the dot fit command. So we are supplying this batch of sequences and the correlated labels. Those labels represent elements. So we are taking the actual behavior of the user and supply it as examples to the fitting mechanism. Now training the network could take time, that's a heavy operation, and we don't want to disturb the user experience. So we are using the fact that the fitting command is asychronical and we can basically do it in the background without disturbing the user. Now the next challenge that we need to solve here is how we should link between those predictions that we are doing and the prefetching that we discussed earlier. So what we want to do is to prefetch content whenever we know that the user is about to hit a certain button, basically making the application faster and faster. And the more that the user uses the app, the faster it gets. So we have this button somewhere in the application that is called show chart. And we can wrap it with this custom component that I call prediction link. And this prediction link will be connected to a react context. The purpose of react context is to supply this global state that is available for the entire app. And here I'm using a react context to encapsulate all the prediction logic. So it will hold all the implementation of the neural network together with the tensorflow code and so on. And it will be available through the entire app. So those prediction link components can query the predictor for the current prediction, the next move of the user. And they can expose this on prediction handler that will be triggered upon an associated prediction. So on prediction, we could use this trigger to communicate with another react context, which is responsible for the prefetching. So it basically acts as a messages bus, announcing that the content that is associated with this button should be prefetched. Now, any consumer that is interested in this information can use it and use dynamic imports to dynamically import the content, the necessary content. So what we're doing here is connecting between predictions and prefetching using two react contexts. Now, let's see how that works in code wise. So we have this button somewhere in the application, this show chart. And we can wrap it with a prediction link, a custom component. Now, this prediction link exposes the on prediction trigger. Whenever there is a prediction for that key chart, then we could do something with this prediction. Now, let's take a closer look on how this prediction link is implemented. We are using the use context hook to connect to the predictor context. And then we are querying this predictor context for the current prediction. If the current prediction matches this specific prediction link, then we are triggering the on prediction handler. Now, in order to have the predictor available through the entire app, we have to wrap the entire app with this higher level provider. Now that we have this prediction link, we can use it to call the prefetch component context using this trigger, basically saying, please prefetch the necessary content that is associated with this button. Now, going back to the component that actually fetch code and render the necessary content, we could again use the use context hook to connect to the prefetcher context, query that context and basically asking if the component that we should prefetch is the chart component. And if so, we're using the dynamic imports to do that. And of course, we can do the same thing with entire routes. So if we have this routing between dashboard and products, we could dynamically import those routes conditionally based on the context, the prefetcher context. So I want to summarize everything that we have discussed here. The first thing that we did was to use the react code splitting api in order to improve the performance of our application. Later, we have combined the lazy loading and the prefetching of content in order to further improve the performance of our application, basically using the time that the user is staring at the screen in order to prepare the next content. Then we showed how we can predict the user behavior using a machine learning based click predictor in the browser environment using tensorflow.js. Of course, predicting the user behavior have more applications that are very interesting to think of. Now, we showed that we can use react context to link between UI handlers and predictions. And finally, we could use another react context to link between predictions and prefetch. So that's the entire overview. That's a personal project that I thought of. And I hope that that will bring you some inspiration about how we can combine AI and UI and basically using machine learning in order to leverage UI development. Thank you. It was fun.