Handling Data at Scale for React Developers

I should I was totally playing that guitar Hi, how are you? Full full from lunch satisfied ready to take in some more knowledge and information and fun react things. Like through your view through your view are awake. For again, how are you feeling? You ready to take in some stuff? Off by one errors, you know. Anyway, hi. I'm nice to nice to see you. I'm Tejas. That's I used to tell people. I used to tell people it's like something but now I say like advantageous. Was contagious anyway, I'm so I I'm the director of developer relations at Zara. Look at this beautiful thing. That is my favorite slide and also one of my five slides I have okay, we're gonna be writing a lot of code in this talk and learning properly this by the way was by Sarah Viara. She's here. She's doing the last talk today so catch that if you want to learn how to do 3D stuff, but that's not what we're here to talk about today. We're here to talk about handling data at scale for react developers handling data at scale for react developers. What does that mean? This is the sounds like a very like abstract marketing talk and the answer is because it's not a marketing talk but it is abstract on purpose. So then I can change it at the last minute as I always do. Okay, but let's get more specific with that and and actually look at a diagram of what we mean by data at scale to do that. I'm going to use an amazing tool called. Excala Draw how many of you heard of excala draw? Yeah, it's if you want to apply Excel draw, yeah, for sure data at scale. This is what it looks like. So usually you have a react app. Or react UI let's say right is the text. Okay. Can you all see? Good. I knew I just asked. and what we're gonna do is usually have a react UI the talks to an API that let's assume out a little bit but then talks to A database okay, and these connections usually look a little bit like this. This may be over simplified. But that is most applications. Is this at scale? Probably not. This is like a single host database and so on at some point you're going to experience like oh we're we're growing and performances important. So what do you do? You probably will distribute your API having a single point of failure is usually a No-No. So what you'll do is you'll do that and you'll have multiple apis that can fetch multiple times and whatever and you can load balance between them. And then okay, you're like, this is cool. But you're going to if you're successful what is successful things do They grow so if you grow you're going to be like, oh, no our database is now the bottleneck. Let's let's make it let's scale it. So you'll scale it vertically and this this vertical scale usually means like adding memory adding disk space adding stuff and it gets quite expensive. You eventually build a super computer or If you want to scale your database the other way you scale horizontally meaning you have maybe a primary instance and some replicas. So when you get data, it spreads out across the replicas and so on and so forth, but then you're going to grow more and we're talking about data at scale. So it's important to establish this context. You're going to grow a little bit more at some point. You're going to be like wait our database is still reading from it is slow and that's because usually database is read from disk disk by Design is not as fast as what Memory, so let's now this I've had this conversation like at least 50 times in the past week. That's a like because I'm a public speaker. I really haven't but you know how it is. So you add some type of like in-memory database just to read from it faster. This will probably be distributed as well. And so now your app will talk to that thing to get data fast and if it's a cache Miss then you read from the database. Okay. This is close to what things look like at scale. I think can't see Dodd's website has something like this in the back. But as you accumulate now data volume, What is the one common feature across things with a ton of data volume? Search so GitHub tiktok Instagram. They eventually when you get enough data, you're going to need search. And so now it gets complicated right because your app can read from the search engine. But it's going to be empty. You're just like okay, nothing's there. You'll need to find a way. to replicate data from your database into your search engine probably in real time would some type of consistency people will use a tool like Kafka or something like this. And so you'll send your data there and this will replicate your data on the search and now everything's connected. Okay, if you want to take a photo of that that is a good representation of what most data infrastructure looks like at scale. Is that what we're going to be talking about at this talk at react Summit? This is what I want to establish with this diagram for the sake of this talk. We're going to make everything with the dashed line. Someone else's problem. This is this is about react. This is what handling data at scale in react. Okay, that is very important that I tell you that. So now I want to talk to you about handling data at scale and react specifically about data fetching. And so that means how can we do this now? There's something to be said for client side rendering server side rendering and so on. What I want to talk to you today about is three ways to fetch data in react this has been talked about a few times. This is nothing new. But before we do that, I need to preface this talk is about what I've learned and what maybe useful for I think it will be useful for you. But you usually want to lean on libraries that are battle tested unit tested even that handle edge cases that we cannot foresee. Is that clear. So the purpose of this talk is not you should be doing it this way use this in production. It's more I want to teach you how stuff works on her just so you understand. So we're all a bit We Come Away with something that we know. Okay. So with that there's three ways to fetch data and react. This is the prevailing three ways. Number one. Probably the most common is you render your component then you fetch you have a component. Use effect fetch dot then set State. This is fairly common. How many of you do that have done that at least one? Yeah. Okay the whole room the next way is fetch then render and you see this in some libraries like next JS. You have like you export get server-side props. You'll fetch on the server and then you render your component. There's something to be said for that. It's arguably faster. Why because you start your fetch way earlier before your component rendering begins. And so then as soon as I promise resolves, you said in state you're good. The Third Way is render as you fetch. This is made possible through the new concurrent features and react 18 and we'll spend the bulk of our time writing code with that specific way to understand it. But as I said, the react team is very clear don't do this in an ad hoc way, but use a library. It's for the best. Okay that said let's just do it now just once okay. So with that that's my last slide. Let's just code. I like coding. inshallah the internet works All right. So we saw the diagram what I want to show you is an app. We'll have an app fetch data in these three ways the app. Let me just start my Dev server here. The app is We're using parcel just because for me it's really great for demos. Oops, SRC slash sorry, it's yarn parcel SRC slash index.html. This is going to bring up our app and what we have is our Jokes app. So these are Developer jokes classic query hit by the one and only Shruti Kapoor by the way shout out. She's a really valuable member of the community and the team and if you don't follow her you should the reason she's on this is because she curates a collection of developer jokes. For example, how do you comfort a JavaScript developer? You console them? Okay, so This is from I just sold it from here. So what we're going to do real quick is fetch data three ways. Okay, we'll do it the way we all know first so our app we have a react app using Create route. So new things we have our app component that renders as you can see the actual elements here the heading the jokes and so on. And lastly we have a joke's component this thing which is supposed to fetch data. but doesn't so we'll fetch data three ways and then we'll talk about what we learn to do that I'm going to Get my data from here and fetch it immediately again inshallah Network, please. Awesome, that's enough. think I think you know what I think I think the network depends. Okay, never mind. It's gonna do something awesome, but now you don't okay, so I'm gonna get a code snippet here in JavaScript fetch this in my app just client side as you know, so. We'll do something like this and when when search changes just refetch. We'll paste the code snippet. I'll add a filter just for fun. where the joke contains search and then instead of console logging the response, I will. Set oops I will not do that. I will set joke. To result dot records. Okay. I'm just fetching data. You're familiar with this. I'm not spending too much time on this and just like that. We have jokes from an API. So what is the most used language in programming profanity? Haha actually funny does the search work we can use words like okay with fee not but okay married life of developer wife right hundred percent. So we're doing we're fetching we're rendering the component then fetching. Okay, this is fairly common if we walk through the code briefly. We have the jokes and state we Fetch and we set State and then everyone's happy notice. We're not handling loading States yet. We'll get to that. This is the traditional way. What about fetch then render you see this less common and user land. I've spoken to some of you and you're like, what is that? Essentially the gist of it is you start fetching early so that your load time is your use and this helps with if you're loading a component with react dot lazy and you're importing it. Dynamically then as soon as the component is imported it fetches immediately and then renders this is actually fast and can't in our case here. We have 20 minutes. So we're going to be working with one network hop, but in cases where you have multiple this really helps you avoid waterfalls Network waterfalls. Meaning you have one request you wait, you have another request and you wait and it's like this waterfall. Okay. This is render than fetch. Let's do fetch then render. So what we're going to do is we're gonna take this whole code snippet thing out of use effect and put it here in module scope and you're like, how can this be? You don't have set jokes exactly. So I'm going to take this piece put it back and use effect. And do promise that then set State. What is promise? It's my fetch call. From module scope. Okay. Does this look odd maybe because you haven't seen this much so we don't get the dynamicism then whatever the dynamicness of the search prop here. So because we're fetching in modulescope doesn't work it does and it's arguably faster. Why is it arguably faster because we're fetching before we render so the data is kind of ready the problem with this of course is it's just going to be stale. I can't control it with props. I can maybe if I made promise like a function like get promise or whatever sure but it's not that common, especially if you're outside a library and the performance benefits are I don't know questionable. let's talk about Render as you fetch. Okay, react 18 has concurrent features where you can render or compute and start rendering a component and when you reach a component that is not yet ready that doesn't yet have data react doesn't render. It renders like a whole fall back and after that it continues down the tree and then when the component is ready then it renders. So it's like render as much as you can fetch while you need it and then render again, okay, that's what we look at the tool that enables. This is called suspense for data fetching. There's an RFC about it here trust the RFC. Don't trust me. Okay. This is this talk is just condensing this with examples, but this is the source of Truth and so is the react blog and the react docs specifically I wanted to show you suspense and data Frameworks. This is important so learn How stuff works with maybe don't do it yourself? Okay, this is just for understanding. Now. How does this suspense for data fetching work? Let's look at an example here by so we already fetching in modulescope. What I want to do is create something called a resource. It's kind of like a primitive that suspense for data fetching will use and this is what they look like. So I'll have a function called create resource. and I'll wrap my entire thing outside of my component in it. Okay? Um, I'll need two pieces of state. So I'll need the result which can be anything. This is typescript and I'll need the status which is in the beginning. It's initial. So I have a bunch of boilerplate here. That's not touching that what I can do is promise. Actually, I just add a DOT then here and by now I have data so data and I'll set my state result is data records. State state status state is done. I'll do the same thing for catch so dot catch. State is error. And the result is in this case. It's not even data. Let me just fix that. Okay, so when the promise resolves we're going to update our scope state in this function. What does this function create resource return? It returns it can really return whatever you want. But in this case, I'm returning an object. with a function called read and what read is going to do is if the state maintained in this function is initial. I can use a switch case for this too. I'm not doing that just because I don't want to if the status is initial so the data hasn't loaded yet. This is this was controversial. So I'm throwing the not yet resolved promise. Who's going to catch it react and then when it's ready you react we'll continue rendering. Okay, if the status is error will throw the result instead and if the status is done, then I write done if the status is done. I just return the result. Return results and so react knows how to deal with this. This is what they handle. Okay, what I'm going to do now after prettier saves, I'm going to create an initial resource. And I will just create it just like that. Okay, and instead of all of this set State stuff all of this. Instead of jokes here. I'm just going to read from my initial resource like that. Save, let's go back to the demo. And it works. So what's the benefit of this? This is great. There's some benefits for example, it doesn't replace. It well, it doesn't actually search we need to figure that out first, right? But look at the loading State. It's instant. There's no like flash more than that, if we throttle the network request just to make this back-end is way too fast. I don't even know what it is. Um, but let's throttle this instead if I can. Yeah, so let's start this here. And now let's actually throttle it a lot. Let's be like three seconds. And now it will notice is we have nothing for three seconds in the whole appears when the jokes part loads. This isn't ideal user experience. So instead what we can do is go to our container app and wrap just the part that needs data. In the suspense component from react. And now if we load the page, wait, wait, wait three second boom so we can granularly control. Where react suspends we can even give it a fallback here with just like that says loading for example. and now if we reload we have loading where we want it. but with suspense you can actually choose different parts of the Dom to hide so We moved it. We now hide the paragraph. we can also hide the input field so you don't type in the loads and So that's this by the way also works on the server side out of the box. No code changes. You shouldn't implement it yourself trust the library but suspense on the server side works exactly like this the HTML actually pauses as it's downloading. The last thing I want to show you is now this doesn't work. Actually, let's make the let's make the search work. So we're already here. What we need to do is we'll use an effect. Such that when search changes. We will recreate the resource. So we'll do resource or just R and set R and we'll use State initial resource. Okay, that is this. Okay. And so now we'll use this instead. So when search changes will set our to search as a parameter and our create resource will accept a search which is a string. And search will be replaced here with search just like that. Okay. This looks great set R is it needs a resource, right? So we create resource in here? Okay. And so now every time the search changes the resource that fetches data changes and our search should work. But it could be that my yeah, my access token expired. I need a new one. The last 10 minutes so that you can't sniff them and steal them. So let's go back to my fetch call here and replace this thing. Okay, cool, and now Oh. I'm doing something wrong, aren't I? Inshallah, the the network isn't working again. Okay. It's 400 right? Sorry. I have I meant joke contains search. Oh, thank you. Whoever said that. Appreciate you there you go default prop and now it should be. On the money. Okay. There we go. But there's a flash look. So now if I try to search this thing if I type like w ah, Hey, so. What about what about I you hide much. So look a keystroke is an urgent update. Changing results is not an urgent update. We need a way to tell react what is an urgent update and what can wait a bit okay to do that. We use another concurrent feature called use transition. This is a hook and this is really important. So pay attention. So you get back a couple loading and start transition from this hook used transition. Okay, what we'll do is we'll wrap this set state of the resource. in this and just like that react now knows I'm gonna wait for all the Urgent stuff and then I'm effect. So let's try this again. Let's type profan. Look at that. Wow. I can type all I want and then there's like a three second delay and then it changes. So that allows us to more fine-grainedly tell react this is important that can wait we can also if we recap more fine-grainedly. Decide where the loader occurs with suspense and that is render as you fetch where react will actually pause where the data is requested and continue down the tree and then when the data is ready come back and render there. These are considerations you're going to have. When you work with data at scale and react why because it can muddy up your UI and introduce Jack. All right, let's recap and finish. What have we learned? We've learned about the three ways to fetch data and react so far right render than fetch fetch the render render as you fetch we looked at them in practice. We didn't get time to look at it in the server side, but that's it's a 20 minute talk but render as you fetch also works on the server side. We also looked at data infrastructure at scale and I don't know I feel like it's been good. So now last question, how do we use this stuff? um You don't do what I showed you here. This is just for understanding. There is a library trust them next JS remix or in framework. I should say you usually want to lean on the shoulders of giants as we all do as Taz said Community is everywhere and with that. I want to wish you happy coding and thank you so much for having me on stage today. Now that was one well deserved Applause. Did you know that I once wrote a whole software to make it look like I'm good at live coding hacker typer and you actually doing it better than my fake software. So that's incredible. All right. We have a few questions from Slidell. So let's get right into it. And also by the way here in in this room, if you are moving between tracks could we do so quietly. So the back of the room can also hear the questions. Thank you very much. All right, so can you use use memo for research replacement when search drink changes to avoid double renders? Probably I don't know. I haven't tried I encourage you try and and let me know. Cool. Is there any similar option to do render as you fetch before react 18 as far as I know? No, I mean in reacts there was some experimental builds before react 18 that you could do that but concurrent features are new and the 18 part is because it's it's a breaking chain so to speak so I don't think so. So all of this stuff that you talked about it's in the RFC. It's in the documentation, right? What's this? That is the RFC now is it ready for production? Use? I I think it's more intent to ship instead of shipped. Right but the intent is there so, okay, cool. What about fetching in an event handler such as on click to navigate? Is this render as you fetch? Could be yeah, I think I don't really know. It's it's fetched then render. Thank you Philip. He was in my workshop. He knows stuff that is so conflict in an event handler is fetch then render, so no. All right. Well, I think these are all the questions that we had from slido. Awesome. So let's give tages one big. Thank you.