Run Games Within Your React Native Apps

Yeah, thanks for having me here. Really excited finally, two years of not being in person conferences to finally be here and one of my favorite conferences. So like really excited to see the audience. And today we're going to talk about some cool stuff. We'll talk about how you can run games within your react Native apps. So, and yeah, clicker is working. Cool. So a bit about myself. So I'm Vladimir Novik. I'm software architect and consultant, and I do a bunch of stuff in all these fields and beyond that. So if you're interested in like something within these fields, you can reach out on Twitter or my website. So let's start with like the first statement that game development is pretty hard. So you need to take care of a lot of things there. Rendering, 3d, materials, lighting, all of this kind of stuff. Physics, how objects interact, how they collide with each other. Is there a gravity, acceleration, all this kind of stuff. AI, how enemies attack your character, how the pathfinding algorithms, all of that. Networking, sound engine, because it's not enough to just have like a regular music in the game. It's all sounds of interactions, of like effects, and even sounds of the footsteps when your character is walking. And when you think of a game, you start with a concept art. So you need to come up with the idea how to do that. And you basically need to get lots and lots of creative assets, which is hours and hours of work of 3d artists. Or you need to purchase them. You get into like level design of things, how you structure each level, how they can be like inherently more complex than one another without actually impacting gameplay. Speaking of which, there's gameplay mechanics, which is different from game to game that you need to take account for. animations, character animations, UI animations, all of that. And obviously post-processing, because effects matter if you have like a bloom effect of like things blooming or HDR and stuff like that. And most of these things are taken care of by game engines. So we need to have one. And there are multiple ones, and the top ones in the market, one of them was actually mentioned in last talk. It's Unity. Now, the other one is Unreal, but today we will talk about Unity game engine and how we can actually create a wrapper for Unity and wrap it into our react Native app. Now, Unity gives us lots and lots of capabilities. It's from their website, it has like 2D, 3d world building, characters, mobile, AR, VR, lots of stuff. And you can actually use all these capabilities and wrap them into your react Native apps nowadays, which is pretty cool, I think. So features for mobile that Unity has, there's a lot of them. I will just briefly go through some of them. So you have like rendering pipeline with like lightning systems and how you present realistic textures on objects and stuff like that. Native 2D tools, if you want to create complex 2D games, you have visual scripting. Now if, let's say, you don't want to write in C sharp, which is the language for Unity is using for development, you can actually use visual tools to really create your game, like full blown game, using visual scripting. Harmonization solutions, AR, which is one of the main reasons to use Unity within react Native because AR within Unity is going like really low level, working with ARKit and ARCore and is really, really performant. And you can check other features here on the link above on what Unity supplies. But today, we will talk about one major feature that is game changer for react Native developers, is using Unity as a library. So react Native and Native Code, they know how to interact with each other, right? So if you look at like really, really simplified version, so you have the javascript, you have the Native Code, and we can use Unity as a library to bridge into this Native Code from Unity side and vice versa, and do the same bridging from the Native to javascript. And that's what we will be doing today. Let's get to the demo, because I want to show you what you can achieve with Unity and react Native. Also with the, focusing also like on performance, we will see like how performant it is. Not the startup though. Takes a little bit of time to load everything. Let's see. Okay, cool. So we have regular react Native app, just blank screen, right? And on the top, you have performance profiler is a little bit small to see, but you can see that we're running with 60 frames per second. Now if I go to Unity side of things, I have two pods. Yeah, sure. Yeah, sure, thanks. Yeah, thanks. And I will also make it bigger. Okay, that's better. So I have react, I have Unity running within react Native app, and on the bottom is react Native, on the top is Unity. So if I change color, you see that immediately the color is changed within the Unity. Now if I write text, I will write, Hi Berlin. We can see that this text is rendered within the Unity, and I can obviously clear that. Now if I click on the message, react Native, ta-da, I have the message within my react Native app. And you can see it's pretty fast. So the whole, like, and you can see the frames on the top, if you've noticed, they drop a little bit, but not that much. Now I can even have post-processing effects here to give some bloom look to the view. But let's get into more complex games. So the character walking, jumping, and even running, a bunch of physics involved, and more complex interaction. So let's see, somewhere it has staircase. So these kind of games, and as you can see on the bottom, there is actually react Native rendered, right? So look at the performance, it's still 60 frames per second. And let me just get to the top. As you can see, we have here our robot walking in the outer space. And it's working pretty good. Now I get back to my react Native app, to a different view, it's a different scene within Unity, and it works like a charm. So let's see how we can actually achieve that. So after the talk, you will have the video of the same demo, so if you want to watch that. And in react Native, it's as simple as just use Unity view, a component that I've created, and we'll see how to create that. And I have on Unity message callback, which basically sets the state within react, and you have these message posted from Unity within the view. Now in order to post something to Unity, I have basically, let me just put the cursor here, or rather, do it that way. I have these function post message, and I use the game object within the Unity editor, its name, and I use the method name of the script that is attached to that game object. And I pass the payload. This payload will be in Unity, basically parsed back into a serialized object, and then I can do whatever I want with it. In my case, I will set the color and set the text. Now in Unity, it will look like this. I have like our interop object on the left, the one that is marked there, and basically, that's like the name of object I'm calling. Now in order to enable this functionality, first in Unity, I need to create these two files. These are two native plugins. That's the option how you can bridge Unity into react Native. You create these plugins. It has to be in assets, plugins, iOS folder, and this example, we will go through iOS example, but the same possible for Android. And then these native files, basically two files. They're the header file and the main file, and we'll get into a little bit of Objective-C and a little bit of C Sharp and a little bit of javascript code. So it will be kind of three languages in the talk, so hopefully you'll find all that. So the first one is this header file and has two functions, basically. It has the register api for native calls, which will be called by a Unity framework, and it will register our send message to mobile app function, and we will be able to call it from Unity. So we basically use XMC and we write our message here under send message to mobile app. We basically make it available for Unity. So that's it. That's the whole thing we need to do from native plugins perspective in Unity. Now sending message to native in Unity will be done in the following way. In Unity, we will use the allele import internal and we will specify the method to be called on the native side. In that case, the Unity framework will search for the method name and will call it. And we'll attach that to a button and call the class native api send message to mobile app. This button has been tapped and that's what you've seen in the demo. Now in Unity, you attach that to the component and you specify the method button clicked press to be called, and that's how the interaction from Unity to react Native works on the Unity side of things. Now receiving message from react Native in Unity, basically as simple as just creating C sharp script with the method that you will be calling from react Native. You've seen you just call the name of the object, the method, and it will be passed to Unity. Now it's up to you what you do there. I just changed the color and the text, but basically you can pass any data and do whatever you want with that. So in order to do these kind of integration, you need to do a few things. And now we will be going through the steps, so you will be able to do that yourself. So first of all in Unity, you need to build your iOS project, and that's the build settings window from Unity that every time you export to iOS, it's just a regular screen you build as normal, nothing different except having these two native plugins, two files for the native plugins. Now I put in my example, I put it under Unity builds iOS folder in react Native project. Now then on react Native project, I will open an Xcode, my regular workspace, react Native, under iOS folder, there is Xcode workspace that I will open. And then just add my Unity project, the build Unity project, into this workspace. So at this point we have two projects, Unity iPhone, which is the default name, and react Native project. Well then, it's really important to actually go to Unity framework, add Unity framework to embedded frameworks under Xcode because we need to use the framework for certain interrupt. Another important step to actually remove linkage of, like linking binary with libraries, you need to remove Unity framework from there, otherwise it will just break and you will see like wild errors. And another helpful thing is to actually drag and drop embedded frameworks before the compile sources because what you want to do, you want to embed framework before actually compiling native files that we'll be writing in a bit. You also need to change data members. Unity iPhone project has data folder which has all like assets and data for the Unity projects. You need to change membership to be Unity framework because when you add the project, it will by default assign membership to our react Native projects, which is something we don't want. So we need to change that. We need to change build products path of Unity framework to point to our build directory because we want everything to be built into one iOS builds directory. And then we also need to find our like native plugins that we wrote and we need to change their membership as well from first of all to Unity framework and then to be public because by default they private. And then we need to build the Unity framework separately. Now these steps absolutely must, you need to go through them, like everything that's on this slide, you need to go through them every time you build your Unity project. And that's a little bit caveat because you need, like if you want to change things in Unity, you basically change things, you rebuild, you do the steps and you get stuff in react Native. So like development process a little bit slower, but you get like lots of benefits from Unity. So in order to integrate with react Native, we will solve a regular integrating native modules the same way we do with, by creating react Native view manager in our example. And we will use RCT export module which is the way to export the module to the javascript side. And we will use RCT export view property which gives us an ability to basically send events to register property on Unity message in our example and send events to javascript. We will create separate class for react Native Unity view. And in this class we'll have the one method that we just had like post message was the message, the method we will, we're calling from javascript. We again will use RCT export method to export that to the javascript side. And we will use Unity post message on the top for actually doing the communication. And we also need to register supported events. Now there is a link how to do that in official docs of react Native and how you actually send events to javascript. Another important, a really important thing that we need to basically make sure we run on the main thread because otherwise we'll be separate thread and we don't want to deal with that. So we use this one method queue dispatch get main queue. Again that's in official docs, how you integrate stuff. In react Native Unity view, it's not the view manager, it's the view naming a little bit off, but we basically send message, we use the send message to go with name. Now this is the default name of Unity framework api to actually send the message to these game object within Unity and call the method on it. And then we have these like wrapper for Unity framework to just make sure it loads and it says the data bundle to be like this data folder, et cetera. Now getting back to javascript side, on javascript side, you remember this Unity view that we had, right? So that's how we write this Unity view. We get UI manager and native synthetic event and we basically require native component and use our view manager as react Native Unity view, like so. Now in post message, we will actually use UI manager, UI manager get view manager config and we post the command to basically send that to Unity. So it will call stuff on the native side. And that's the way how we use that. Now let's get some really cool news. Now that can be a little bit complex, right? To write all these like native code and stuff. So there is a library for that. It's called react Native Unity. Ta-da! And so no need to write native code, right? So that's like a huge part of things. You just use react Native Unity. It has Unity view. It supports Android as well. So I just described how to do it with iOS. You still need to do all the setup and build steps. So it's really good now you know how to actually, how it actually works under the hood. So there are some helpful links that you can screenshot and get. There is a demo code in this repo. In order to learn Unity, you can go to learnunity.com. There are amazing paths to actually learn Unity development. There is a Unity asset store where you can purchase assets for like fraction of the cost. And if you have like any questions for me after the talk, et cetera, so reach out on Twitter. That's my Twitter on the top. And you can like ping me for consulting on my website. And I also have these like forum. I will be doing like workshop and courses in upcoming years. So if you're interested in learning more about like when it will be done, you can like just fill it in and I will, I promise not to spam. I will just send you an email whenever any course or workshop will be ready. And I think at this point the clicker stopped working and the reason for that is I'm kind of done. And I think we have questions, several minutes for questions here. And thanks a lot for listening. Thank you. Yeah. Thank you very much Vladimir. Yeah. Just go ahead. Come, come, come, come. We do have some questions though. Yeah. Are we running short on time? Almost. Yeah. So the first question, what would be a real world use case for integrating Unity in react Native? I saw it coming. That's like, yeah. The real world case scenario nowadays, mostly it's AR. Because if you want to integrate AR into your react Native apps right now, it's either you use like other frameworks such as Viro for instance, which is good one as well. But if you want more like native capabilities and getting into all the capabilities ARKit has to offer or ARCore, Unity has AR Foundation Framework, which enables you to leverage almost everything you can do with ARKit. And now you can bridge that into Unity. And that's like one of the use cases. There are quite a lot. I mean, you can do gamification, you can do like fancy ads within. I know like we all hate ads, but I mean, sometimes it's driving business. Yeah, I've heard that the Unity ads are pretty good. Yeah, you can use like ads, you can use like some 3d views, which are like really performance oriented. For simple games, the ToggleView before mine was pretty good, like to just try things in for like 2D games and publish them as a web view. I pretty like Unity capabilities in general. That's actually a good answer because you also covered another question, which if AR is actually possible, so you already addressed that. There's another follow up question to the first one. What would be a really like a use case for Unity inside Reagnative? Like you showcase like fireworks and things like that. But what do you think in your perspective, it's like a really real use case of Unity? So like apart from AR, I mean, like, yes, apart from AR, like just what do you think about UI libraries that already exist in Unity? Can we leverage those? Yeah, definitely. Like sometimes and that's good you bring it up because Unity assets store has lots of amazing assets that you can purchase and some of those are really like fancy UI libraries with a bunch of effects and fireworks. So if you want to have in general something like physics related, and I think like particles are like also like physics related, like fireworks and stuff, you can use these assets from assets store and embed them within react Native app. And as you've seen, you're getting pretty good performance on that. Great. So let's talk about performance. How performant is such UI and fast in game interaction scenarios, especially given that on every call the JSON serialization and deserialization is included? So basically, you're not introducing another bridge here, you're basically in the same world as in general, like react Native, because basically what you're doing is serializing the same data and then passing that through the bridge. And that you call in native methods, then go into Unity. So there is no like another layer here. So performance will be pretty much the same, like talking of like passing through the bridge will pretty much the same as regular react Native apps. With that said, if you want to make things more performant, I would think of like serializing the data, passing to the Unity and handing over the logic to Unity to do like the whole animations and stuff and then passing the data back. So I would, for instance, here I had like lots of renders and passing things through Unity. That's for the sake of the demo, but I would advise against that because that's, as you mentioned, there's lots of traffic through the bridge, you don't want that. Yeah, the thread will be bloated anyway. So I think that's one down or drawback of react Native. But I think that the new architecture is going to solve some parts of that as well. Speaking about native modules, have you ever thought about Expo native modules, like to expose a Unity view using Expo native modules? What do you mean, like exposing? Like Expo has created these native modules. So basically with Expo, yeah, I've thought about that, like something I haven't tried, you need to experiment with that a bit. Just to bring in some context, Unity at some point, they had like really like vendor log sort of thing. They didn't have this feature, Unity as a library, it's a fairly new one, relatively, as things go slower in the game development world, it's like for a couple of years. But it's still like from Unity perspective, it's a relatively new one. So there is no much experimentation done on that regard, especially with react Native. But yeah, that's definitely something to look into and try it out. And we do have one more question. Why not use Unity and build iOS without additional wrappers? What is the profit itself? If just using Unity? Yeah, without going through react Native, what do you think is the benefit here? I think that you need to identify what's your app use case. So it's a regular statement from consultant, it depends, right? So usually, I would say if your app is really heavy oriented into fancy UI and rendering and physics and stuff like that, there is no reason to use react Native. But in lots of cases, we see that clients with already existing react Native apps, they want to introduce more gaming capabilities and then to some degree, it limits their imagination, I would say. If they want to see something fancy, they cannot do that with just typescript and javascript, they need to do something more game-ish. And in that case, Unity and react Native will be the good choice. If it's something more simple, then it's either react Native or Unity. But it's good to know that the integration exists and it opens up a lot of opportunities for experimentation and actual projects. Speaking about integration, have you ever thought about the size of the bundle after integrating Unity? Yeah, the bundle will be absolutely, I wouldn't say huge, it will be way higher because obviously Unity, with Unity you need to deal with all the 3d assets if you're dealing with the game. I mean, this demo is I think like 150 MB bundle, which is pretty huge. And if you think of game development in general, usually they have huge bundles. So that's something to take into account, obviously. Okay, so it's basically something new that is going to enhance over time, but now we should also think about trees and nature, just saying, because 150 MB is just for a Hello World or a 3d box, it's quite a lot, but definitely maybe it's going to improve, right? There is optimization. And speaking of that, there is an option with Unity to use asset bundles. So basically you don't download all the assets, you get them through the network on the fly. I mean, there is capability to do that. And that's something obviously to take into account when you develop your app. You don't want to get, especially like with AR, for instance, because you have all these like 3d models, which is like really like taking some space on the device. Okay, thank you. I think we're running out of time. So yeah, let's give a huge shout out to Vladimir. Thank you very much. Thanks.