React Summit

React Summit 2022

Marrying WASM/WebGL Games with React UI

Hey everyone, it's Jonny and today I'm talking about marrying WebAssembly and WebGL games with React, or in some other words, using the right technology for the right job. So first about me, I work at a social media company as a web engineer. That media company has a blue logo, but it isn't Facebook. In my free time, I love to ride bicycles. That can be helping my brothers with bicycle advertisements, can be a road bike, it can be a mountain bike, as long as it's bicycle related, I probably like it. And the other big hobby I have is games. Together with my dad, I've been making multiple board games, for example Lost Valley, North, Half-Band Heroes, they're usually funded via some crowdfunding campaigns. And since high school, I've also been developing multiple electronic games. Initially, they were all iOS based. Recently, I've been branching out a bit, as you might notice from the talk, I'm talking about making games on the web. Why did I become an engineer? I became an engineer to save time, to save my time and to save other people's time, so they have more time available to play games. And that brings me to the first thing I want to talk about, picking the right tool for the job. My previous games, I largely made with Cocos2D, but Cocos2D is a very code driven game engine. So it's familiar for a developer to just write out the code, but it's not very time efficient, because there's no tooling. And recently I started using the Godot game engine, which is enabling me to save a lot of time on the web development side. I love to use React, which is also allowing me to save a lot of time. And I intentionally put it on the board game side here, because there's one interesting similarity in board game development and web development with React. And that is being declarative. If I come up with a new rule to play on the table with my other testers, I just roughly explain them the rule and declare my goal, my intention, what is this rule change supposed to do? And then they will interpret it and use their brain processing power to come up with an interpretation how to actually move it into the game system, how to execute the rule. On the other hand, if I want to teach a new rule to the computer to play games with, I need to be very precise about everything. So let's ask the question, what's the quickest way to make a game and why might React be a good choice for that? And why might Godot be a good choice for that? And also very importantly, what's the most fun way to make a game? Because in the end, this is a talk with no commercial interest. This is just about having fun developing some games. Okay, so why React? Some people might remember my talk from React Summit Remote Edition in 2020, where I talked about making Diagonal 4. And the interesting thing about that game is, first, it's very declarative. I just have a very simple discrete board state. In the end, it's just a bit field of eight by seven pieces. And I can just go from that bit field and I can calculate all available moves for every player and I can just calculate the view very directly, because it's just a very simple state and very simple rendering. So that maps well to React. The other great advantage of using React is that it's very easy to make a responsive UI and to have some design systems making that even easier on the web. Why might React be not the best choice to make a game or modern game on the web? Let's look at this example, Blood Fever. It's a game of a vegetarian vampire defending his coffin against zombies. But the important point is that there might be hundreds of enemies who all are running animations on their own. They don't have a very discrete position. It's more like a continuous state. Some functional reactive programming privates might say it's still a state and you can still do very atomic calculations to come up with possible actions by the player and then put that in a very big state machine. On the other hand, with animations, it's just not the best thing, I would say, to keep a nice performance running. So just the performance aspect, React might not be the best choice here. And also tooling. Game dev gets a lot quicker if you use the correct tooling and React is really not well equipped there. The ecosystem works. So who might bring good tooling to the table? Godot. Why use Godot for game development? First thing first, it's free software. It's very popular software. If you look at this graph on OSS inside on the right, this is the pull request creators and you just see Godot becoming the biggest player with most contributors by far in the game engine space. It exports to web, iOS, Android, Windows, Linux, OSX, pretty much all platforms you could need. And it is a very mighty editor. If you look at the screenshot on the left, you see the scene graph. On the right, you see the scene in the editor and the node editor. So you can click simple things and you can move them around, which really makes a lot quicker than doing all of that just via code. And lastly, it brings an integrated scripting language called Godot script, which is very Python-esque and might be simple to pick up for a lot of people. Some other people might still miss TypeScript, but there are solutions for that. But that's a story for another time. Why should you maybe not use Godot? Well, it doesn't have any flex block or grid things. So layouts might get a bit complicated. Why could that be an issue? Just look at this example from Age of Empires 3. And you see a screenshot and the screen was very big, but the menu didn't scale. So they do some active background images. It's just not that great. It can get even worse in-game if you have an even wider screen and where the person would naturally just look in the middle of the screen and only see that. And the menus left and right are very far away. You might need to physically turn your head. So it's just very awful UI-wise, because the tuning isn't that great or made for it. Another reason to not use Godot is there might be some external SDKs missing. One great thing about the web is that it's very easy to just go to NPM and integrate some modules into your application. It's a bit more complicated with Godot. Also, on the web, you could just easily integrate some authentication SDK from Twitter or from Facebook and then add some sharing functionality. Whereas doing that natively in the game engine itself would be a lot more complicated. And lastly, I'm a web engineer most of the time, so I just might miss some web tooling if I just use the game engine. I also want to be up to date with web stuff, so it's a fun way to play around. That said, it's also about having fun, this talk. What's the solution? The obvious solution is combining React with Godot, but how would we approach that? Well, first caveat, non-commercial. This talk, as I said, there's no commercial interest. This is just me in my hobby time playing around with games. So if you want to make money with your games, you might have widely different considerations to think about. The other caveat is just a prototype. I just did start developing this like two months ago and was thinking about, hey, how can I combine my two favorite technologies? Came up with this, so I'm showing it at the moment. And let's just jump into, well, the example video. So it's all running on a website. The website is just, in this case, host on localhost. It's Johnny's Playground, and there's another game running on the localhost. So there's my game called Kellergewurbel live, which is just a dungeon runner. And you see the connection between the React UI and the game running itself. So I changed my name. Johnny is now popping up in the game itself. What is the game about? You're just running in that dungeon. You can push your adventurer up and down, and you need to avoid all the dungeon dwellers and just not touch them. Now I did touch the developers, so the game is restarting. I think that wasn't quick enough, so let's use the React UI to change the game speed. And well, that might have been slightly too quick, so let's slow it down a bit again and then speed it up again just to see the connection is instantious, so it's not waiting for the game to reload again to pick up. The status changed from the React UI, but it's like a full connection. And well, let's talk about a very annoying thing, sharing scores on social media. We might have all been annoyed by the word craze and people spamming our Twitter feeds, but this game also can generate a tweet for you. You are authenticated in the background on Twitter. You can press share, and it will be automatically shared to your feed. It's just one possibility of the web integration. But let's look at one other feature. Let's switch games dynamically on the same website. Now I'm switching to the remotely hosted version of Calligraphic Velvet Live, which has a different scaling mode activated, so now it's just using the design size of the game. So it's just 480 x 320. And lastly, it switched to yet another game, Blood Fever, to just show that there's also some other UI patterns possible. The game before has been using the keyboard, and with Blood Fever, all is mouse-controlled now, and it's all still picking up the context nicely on the web. And that's it for a quick demo combining React with GoDo games on the web. How do you get there? Well, the first step is actually have a game in the GoDo engine. Calligraphic Velvet Live, as I talked about, is a quick 2D dungeon runner. It's all available on GitHub under GPL license if you want to check it out. So you just create that game in GoDo. You can debug it in GoDo. How do you export it? Well, GoDo has a very simple export dialog. You just say you want to export it to the web in the HTML5 version. You press export, and we'll generate some files. We'll actually generate this doc folder on the left. So there's going to be the game HTML. This is GoDo's native game wrapper, but that game wrapper really only supports running the game standalone on the website, and it doesn't really have a nice environment to add any further UI on. There's also a file called game.js, which contains some of the game code and some of the engine code, but we're going to look into details for that slightly later. And there is a game.wasm file and a game.pak file. The game.wasm file is the actual engine compiled down to WebAssembly and also some of your game code compiled down, depending on how you configure the export. And the game.pak file is basically all the resources you need for your game, which most of the time are going to be textures or audio files. Okay, so GoDo exported that, but how can we use that? Let's take a step back initially. Let's quickly talk about WebAssembly and WebGL. That might be relevant for you. WebAssembly, there's this long description you can read if you want to, but roughly speaking, it's just a binary instruction format, which applications can compile to if they don't want to, for example, compile to JavaScript or some other weird transformation, but it's more like a binary format. And that's very common for native engines to now add another compilation target. So GoDo engine can just choose WebAssembly as another compilation target and then can compile the C++ engine code to that. And WebGL is the JavaScript API for rendering 3D and 2D graphics. I personally love to do 2D games because it's just easier as a single developer to just pick up some graphics from like OpenGameArt on the internet for free and use them in their own games. It's just a lot quicker than 3D, but you could also, with the technology presented to you, just build a 3D game if you wanted to. And WebGL also means you're rendering into this canvas. That means there's no further DOM elements for the game itself. It's all like a pixel target. It's rendered into the texture, and the texture's rendered in the canvas. Okay, enough for that excuse. Let's jump back into the Game.js file. Game.js file actually considers largely of two bits. The first bit, which looks very horrifying, that's why I'm not zooming in, is in the end just mapping some of the JavaScript code to some of the WebAssembly code so the connection back and forth works out. But we can totally ignore that because it doesn't really matter at all for embedding our games. And there also is some engine code. That engine code is loading the fetch files. That's a bit more relevant for us because the Godot wrapper I'm presenting is extracting some of that engine code. But now you might ask, okay, so Godot is providing this fully. Why can't we just use an iframe? Yes, you could use an iframe, but you also can't use an iframe because you might have some problem with calls because you're going to run into situations where you can't run the code. And if you want to, you might run into a situation where you can't pass messages to the iframe because it's blocked. Also, it's just using iframe isn't really the React approach, I would say. So what's our local architecture going to be? Well, first things first, the user visits the website. The website renders some UI. And that website UI also renders the gsgd host, which is provided in an on-prem package. That gsgd host then goes to some other web host where you uploaded the game.js game pack and game.wasm file, which Godot exported. And then the host is loading that JavaScript into the local context. It's picking up the engine definition from the game.js file. It's instantiating that. And it's also providing a WebGL context. And then the engine starts running and is rendering into the WebGL context that the user can interact with. What is the infrastructure for that, because it sounds very horrible to set all of that up for all the games you want to publish. Also, again, I think a very convenient and fun solution for that, you just have an embedding website you want to work on. You have a Godot project you want to work on. That Godot project just pushes to a game repo on GitHub. On GitHub in a YAML file, you can reuse the workflows I'm providing in the ksgodot-ci package. These workflows will use a Docker image to compile the game on the CI with the web target. And they will then be published to the game page. And you can run like your normal CI on your web repo as you're used to. And then just your web page is configured to fetch the game data files from the game page and then can run it from there. Next step, you might ask, can we actually look at the code? Absolutely. It's all available online. I can also, well, share a screenshot to show the very fundamentals, but I don't really have time to go into the details. So let's look at a higher level abstraction here. What is the actual communication happening between React and the Godot game running on your website? So on the website component, you have a host component. That host component has props of a player name and a power up. It has a callback of like on report score to trigger that score sharing pop up. The host will share these properties through the window object because there needs to be like some back and forth and sharing like the callbacks with the game. The Godot game itself will look on the window object, see, OK, what's my initial player name? What's my initial power up value? And then we'll also set these two functions on player name change and on player power up change on the window. If the props change on your component, these callbacks will be called. And then there's signals on the Godot side. So player name change signal, which will trigger if you want to change the name of the power up. And then the other game components can read the signals. How does it look? Very simple. In the end, you're just registering a callback to say, OK, when the power up change, let's call that function. In this case, this is changing like some of the particle effects slightly. And that's actually most of it from like the technical perspective. So what's working well at the moment? Working well is loading and caching games. It's pretty much feature complete. It's quick. You'd go back to the website, you will get the cached version. So running fine. The communication between React and Godot is working flawlessly. As you did see in the example, it's just going back and forth. Obviously, it could be abstracted slightly more, but about that in a moment. And in general, it's very easy to add new games to your own personal game playing playground. And I would say the developer experience is very nice. I can just create a new game repo. I reuse my workflows, and it's pretty much running on GitHub instantly. So that's all pretty nice. And most important thing, it's a lot of fun because you can use your full game engine, and you can still reuse the games in your personal React website. So take new technology and technology you're used to. What could be improved in the future? Well, garbage collection isn't perfect yet because sometimes the game engine kind of sticks around and is taking up some memory. So I need to look into that. There could be more integrations integrated, like, for example, Twitter SDK to share tweets, or an ad SDK if you actually want to make money with the games, stuff like that. And the message passing between Godot and the game could be abstracted a bit further. You might notice that's all not that fun bit, so I didn't look into that, but I will look into that in the future. Last question, want to try? It's all available on GitHub at github.com. There's different repos you can look in. There's two example games. There's a CI package. There's actually a GSGD bridge package, which is providing the talked about React component. And if you have any further questions, feel free to reach out to me at chaosyogo on Twitter. And thank you very much for listening.


Johannes Goslar

JS GameDev Summit

JS GameDev Summit 2022

Unreal Engine in WebAssembly/WebGPU

Alex St. Louis

Extending Unity WebGL With Javascript

Brendan Duncan

Detect and Avoid Common Performance and Memory Issues in Unity WebGL Builds

♪♪ Hello everyone, my name is Oz and I'm a Game Dev Evangelist at Backtrace Sauce Labs. Today we're going to see some of the ways we can avoid common performance and memory issues arising in Unity WebGL builds. The first point we'll be taking a look into is managing performance. And a major constraint running a WebGL build in a browser with good performance is memory. So in terms of memory, what are the problems and what are the considerations that we got to take a look at? So as you know, the complexity of the content you can run is constrained by memory. And the memory is allocated by the browser and the amount of memory it has available varies on a number of factors, including the device you use, the operating system you use, which browser you use, and whether it runs on a 32-bit or a 62-bit browser, and a couple of other factors. So to understand how memory impacts performance, it is very important to first see how memory is allocated in a Unity WebGL build. For this, we need to take a look into Unity Heap. So what is Unity Heap, or a Memory Heap, as Unity calls it? So basically, this is where all runtime objects are stored, and these could include native objects, Assets that are loaded, Scenes, Shaders, Animation Files, as well as other managed objects. So it is worth noting that this Heap is one contiguous block of allocated memory. Until a few years ago, you had to allocate the max size of the Heap in the build settings, but now Unity supports Heap expansion on demand, depending on the needs of the game, and it's expandable to up to two games. However, the same feature can actually often cause your game to crash, especially where the browser fails to allocate one contiguous block of memory. And this is exactly why it's very important to keep the Heap size as small as possible. Now, to understand how we can keep the Memory Heap size small, we have to first take a look into what Unity does to all of the Assets and Scene data in a project, and what happens when you build a WebGL application in Unity. So when you build a WebGL application, what happens is Unity generates a.data file. And this is basically all the Assets and Scenes in the application that is required at the time of launch, and all of that is packed into a.data file, including Unity Scenes, Textures, Models, UI Sprites, Audio Assets, Shaders, and pretty much everything else that you need for the game to run. And basically, Unity WebGL does not have access to a real file system. Therefore, it has to create a virtual memory file system, and the browser then unpacks the.data file into this particular virtual file system. And the browser, while the game is running, keeps the data uncompressed. Now, imagine if you have a complex Scene with all sorts of Assets, including 3D Models, Shaders, everything. You can see how possibly it can run into memory issues and slow down the game performance as well. So the question then that arises is, what can you do to reduce memory? Well, there are a few techniques that you can adopt to reduce memory use. And one way, the most common, is to use Asset Bundle. So you could put all of the most frequently, but bigger Assets into an Asset Bundle, and load it, unload it, on demand, whenever you want. And it's also worth noting that the Asset Bundles are directly downloaded into Unity's heap, and therefore, they avoid extra allocation done by a browser. If there are big bundles, one technique is to possibly cache the data. And this is basically one additional thing that helps you to avoid re-downloading each time you actually play the game. So the third one is something called an Addressable System, which is an alternative to Asset Bundles. And sometimes it's touted as a better version of Asset Bundles. However, it does have some of its own share of problems, including supporting WebGL builds out of the box. But there are some workarounds that exist to make it work. The next important consideration with regards to WebGL builds is the Garbage Collection, or GC in short. So typically, what GC does is it locates and collects unused memory, and then reallocates it into the Unity heap. On other platforms, when GC runs, all running threads are paused to give time for the GC to check the stack. But this is not possible on WebGL due to limitations in JavaScript. So what Unity does here is it simply runs once after each frame, because the condition for the GC to run on WebGL is that the stack has to be empty, and this happens only after each frame is rendered. Now, we'll look into some of the examples of code that increase GC and reduce performance, and how we can actually write a better code. So in the first example that I have here, you can see how manipulating a string is a major cause of performance issues, and sometimes can also cause an application to run out of memory. And the primary reason this happens is because any manipulation of string causes Unity to create a new string each time, and this can quickly add up. For instance, if the code were to be part of the Abyte loop. This example you can see here is where you have a countdown timer showing on a UI, and the countdown timer text is being updated continuously in the Abyte loop by appending the countdown prefix with the countdown value. And each time you do that, it's creating a new string, which can quickly really add up, and is not good for the memory. On the right, you can see how this code is being rewritten into a better code, where we separate the label and the values, therefore avoid the string manipulation altogether. Here's another example of string manipulation. If it's used in large for a loop. In a non-WebGL build, this might run as fine as it could be, because the garbage collector will be involved, but not so much in a WebGL build. Because in a WebGL build, it doesn't get a chance to get involved until the end of the frame, and therefore this will likely run out of memory and crash. Here I do have some other examples where you can optimize Unity code to reduce memory allocation and reduce the need for frequent garbage collection. When using strings, for instance, if you need to manipulate or append string, use StringBuilder instead of Plus. If you're using Unity functions, where the function will return an array and you have to do some array manipulations or iterate through an array, you should cache it before you use it in a for loop. For comparisons, like comparing a name or a tag, not many of us know that these accessors actually return a new string each time, so it's better to use, for instance, to compare to a tag, use CompareTag function. And last but not the least, in case of coroutines, one common thing we tend to do as programmers is to use a line that returns new waitForSeconds or waitUntil or waitWhile. So instead, cache it and use that, especially if you're using it in a for loop. While we spoke about the ways memory can be managed in some situations and how we can reduce memory use, there are some unexpected situations that can cause applications to run out of memory or cause applications to even crash. So this is where tools that can help debugging these issues come into play. So one of those tools is Backtrace, which I use in applications and games I make for any platform, and of course, WebGL is no exception. So when I integrate that, I can see errors in real time and dashboard and help me troubleshoot performance and out-of-memory issues. Things like what we see today, these can be actually used, and then that information can be collected to fix further problems before they get out of hand. That's pretty much the end of the session. If you have any questions about WebGL or Unity, please feel free to get in touch with me. My contact details are on this slide. Thanks for tuning in. ♪♪♪


#webgl