Marrying WASM/WebGL Games with React UI

Hey everyone, it's Jonny and today I'm talking about marrying WebAssembly and WebGL games with React, or in some other words, using the right technology for the right job. So first about me, I work at a social media company as a web engineer. That media company has a blue logo, but it isn't Facebook. In my free time, I love to ride bicycles. That can be helping my brothers with bicycle advertisements, can be a road bike, it can be a mountain bike, as long as it's bicycle related, I probably like it. And the other big hobby I have is games. Together with my dad, I've been making multiple board games, for example Lost Valley, North, Half-Band Heroes, they're usually funded via some crowdfunding campaigns. And since high school, I've also been developing multiple electronic games. Initially, they were all iOS based. Recently, I've been branching out a bit, as you might notice from the talk, I'm talking about making games on the web. Why did I become an engineer? I became an engineer to save time, to save my time and to save other people's time, so they have more time available to play games. And that brings me to the first thing I want to talk about, picking the right tool for the job. My previous games, I largely made with Cocos2D, but Cocos2D is a very code driven game engine. So it's familiar for a developer to just write out the code, but it's not very time efficient, because there's no tooling. And recently I started using the Godot game engine, which is enabling me to save a lot of time on the web development side. I love to use React, which is also allowing me to save a lot of time. And I intentionally put it on the board game side here, because there's one interesting similarity in board game development and web development with React. And that is being declarative. If I come up with a new rule to play on the table with my other testers, I just roughly explain them the rule and declare my goal, my intention, what is this rule change supposed to do? And then they will interpret it and use their brain processing power to come up with an interpretation how to actually move it into the game system, how to execute the rule. On the other hand, if I want to teach a new rule to the computer to play games with, I need to be very precise about everything. So let's ask the question, what's the quickest way to make a game and why might React be a good choice for that? And why might Godot be a good choice for that? And also very importantly, what's the most fun way to make a game? Because in the end, this is a talk with no commercial interest. This is just about having fun developing some games. Okay, so why React? Some people might remember my talk from React Summit Remote Edition in 2020, where I talked about making Diagonal 4. And the interesting thing about that game is, first, it's very declarative. I just have a very simple discrete board state. In the end, it's just a bit field of eight by seven pieces. And I can just go from that bit field and I can calculate all available moves for every player and I can just calculate the view very directly, because it's just a very simple state and very simple rendering. So that maps well to React. The other great advantage of using React is that it's very easy to make a responsive UI and to have some design systems making that even easier on the web. Why might React be not the best choice to make a game or modern game on the web? Let's look at this example, Blood Fever. It's a game of a vegetarian vampire defending his coffin against zombies. But the important point is that there might be hundreds of enemies who all are running animations on their own. They don't have a very discrete position. It's more like a continuous state. Some functional reactive programming privates might say it's still a state and you can still do very atomic calculations to come up with possible actions by the player and then put that in a very big state machine. On the other hand, with animations, it's just not the best thing, I would say, to keep a nice performance running. So just the performance aspect, React might not be the best choice here. And also tooling. Game dev gets a lot quicker if you use the correct tooling and React is really not well equipped there. The ecosystem works. So who might bring good tooling to the table? Godot. Why use Godot for game development? First thing first, it's free software. It's very popular software. If you look at this graph on OSS inside on the right, this is the pull request creators and you just see Godot becoming the biggest player with most contributors by far in the game engine space. It exports to web, iOS, Android, Windows, Linux, OSX, pretty much all platforms you could need. And it is a very mighty editor. If you look at the screenshot on the left, you see the scene graph. On the right, you see the scene in the editor and the node editor. So you can click simple things and you can move them around, which really makes a lot quicker than doing all of that just via code. And lastly, it brings an integrated scripting language called Godot script, which is very Python-esque and might be simple to pick up for a lot of people. Some other people might still miss TypeScript, but there are solutions for that. But that's a story for another time. Why should you maybe not use Godot? Well, it doesn't have any flex block or grid things. So layouts might get a bit complicated. Why could that be an issue? Just look at this example from Age of Empires 3. And you see a screenshot and the screen was very big, but the menu didn't scale. So they do some active background images. It's just not that great. It can get even worse in-game if you have an even wider screen and where the person would naturally just look in the middle of the screen and only see that. And the menus left and right are very far away. You might need to physically turn your head. So it's just very awful UI-wise, because the tuning isn't that great or made for it. Another reason to not use Godot is there might be some external SDKs missing. One great thing about the web is that it's very easy to just go to NPM and integrate some modules into your application. It's a bit more complicated with Godot. Also, on the web, you could just easily integrate some authentication SDK from Twitter or from Facebook and then add some sharing functionality. Whereas doing that natively in the game engine itself would be a lot more complicated. And lastly, I'm a web engineer most of the time, so I just might miss some web tooling if I just use the game engine. I also want to be up to date with web stuff, so it's a fun way to play around. That said, it's also about having fun, this talk. What's the solution? The obvious solution is combining React with Godot, but how would we approach that? Well, first caveat, non-commercial. This talk, as I said, there's no commercial interest. This is just me in my hobby time playing around with games. So if you want to make money with your games, you might have widely different considerations to think about. The other caveat is just a prototype. I just did start developing this like two months ago and was thinking about, hey, how can I combine my two favorite technologies? Came up with this, so I'm showing it at the moment. And let's just jump into, well, the example video. So it's all running on a website. The website is just, in this case, host on localhost. It's Johnny's Playground, and there's another game running on the localhost. So there's my game called Kellergewurbel live, which is just a dungeon runner. And you see the connection between the React UI and the game running itself. So I changed my name. Johnny is now popping up in the game itself. What is the game about? You're just running in that dungeon. You can push your adventurer up and down, and you need to avoid all the dungeon dwellers and just not touch them. Now I did touch the developers, so the game is restarting. I think that wasn't quick enough, so let's use the React UI to change the game speed. And well, that might have been slightly too quick, so let's slow it down a bit again and then speed it up again just to see the connection is instantious, so it's not waiting for the game to reload again to pick up. The status changed from the React UI, but it's like a full connection. And well, let's talk about a very annoying thing, sharing scores on social media. We might have all been annoyed by the word craze and people spamming our Twitter feeds, but this game also can generate a tweet for you. You are authenticated in the background on Twitter. You can press share, and it will be automatically shared to your feed. It's just one possibility of the web integration. But let's look at one other feature. Let's switch games dynamically on the same website. Now I'm switching to the remotely hosted version of Calligraphic Velvet Live, which has a different scaling mode activated, so now it's just using the design size of the game. So it's just 480 x 320. And lastly, it switched to yet another game, Blood Fever, to just show that there's also some other UI patterns possible. The game before has been using the keyboard, and with Blood Fever, all is mouse-controlled now, and it's all still picking up the context nicely on the web. And that's it for a quick demo combining React with GoDo games on the web. How do you get there? Well, the first step is actually have a game in the GoDo engine. Calligraphic Velvet Live, as I talked about, is a quick 2D dungeon runner. It's all available on GitHub under GPL license if you want to check it out. So you just create that game in GoDo. You can debug it in GoDo. How do you export it? Well, GoDo has a very simple export dialog. You just say you want to export it to the web in the HTML5 version. You press export, and we'll generate some files. We'll actually generate this doc folder on the left. So there's going to be the game HTML. This is GoDo's native game wrapper, but that game wrapper really only supports running the game standalone on the website, and it doesn't really have a nice environment to add any further UI on. There's also a file called game.js, which contains some of the game code and some of the engine code, but we're going to look into details for that slightly later. And there is a game.wasm file and a game.pak file. The game.wasm file is the actual engine compiled down to WebAssembly and also some of your game code compiled down, depending on how you configure the export. And the game.pak file is basically all the resources you need for your game, which most of the time are going to be textures or audio files. Okay, so GoDo exported that, but how can we use that? Let's take a step back initially. Let's quickly talk about WebAssembly and WebGL. That might be relevant for you. WebAssembly, there's this long description you can read if you want to, but roughly speaking, it's just a binary instruction format, which applications can compile to if they don't want to, for example, compile to JavaScript or some other weird transformation, but it's more like a binary format. And that's very common for native engines to now add another compilation target. So GoDo engine can just choose WebAssembly as another compilation target and then can compile the C++ engine code to that. And WebGL is the JavaScript API for rendering 3D and 2D graphics. I personally love to do 2D games because it's just easier as a single developer to just pick up some graphics from like OpenGameArt on the internet for free and use them in their own games. It's just a lot quicker than 3D, but you could also, with the technology presented to you, just build a 3D game if you wanted to. And WebGL also means you're rendering into this canvas. That means there's no further DOM elements for the game itself. It's all like a pixel target. It's rendered into the texture, and the texture's rendered in the canvas. Okay, enough for that excuse. Let's jump back into the Game.js file. Game.js file actually considers largely of two bits. The first bit, which looks very horrifying, that's why I'm not zooming in, is in the end just mapping some of the JavaScript code to some of the WebAssembly code so the connection back and forth works out. But we can totally ignore that because it doesn't really matter at all for embedding our games. And there also is some engine code. That engine code is loading the fetch files. That's a bit more relevant for us because the Godot wrapper I'm presenting is extracting some of that engine code. But now you might ask, okay, so Godot is providing this fully. Why can't we just use an iframe? Yes, you could use an iframe, but you also can't use an iframe because you might have some problem with calls because you're going to run into situations where you can't run the code. And if you want to, you might run into a situation where you can't pass messages to the iframe because it's blocked. Also, it's just using iframe isn't really the React approach, I would say. So what's our local architecture going to be? Well, first things first, the user visits the website. The website renders some UI. And that website UI also renders the gsgd host, which is provided in an on-prem package. That gsgd host then goes to some other web host where you uploaded the game.js game pack and game.wasm file, which Godot exported. And then the host is loading that JavaScript into the local context. It's picking up the engine definition from the game.js file. It's instantiating that. And it's also providing a WebGL context. And then the engine starts running and is rendering into the WebGL context that the user can interact with. What is the infrastructure for that, because it sounds very horrible to set all of that up for all the games you want to publish. Also, again, I think a very convenient and fun solution for that, you just have an embedding website you want to work on. You have a Godot project you want to work on. That Godot project just pushes to a game repo on GitHub. On GitHub in a YAML file, you can reuse the workflows I'm providing in the ksgodot-ci package. These workflows will use a Docker image to compile the game on the CI with the web target. And they will then be published to the game page. And you can run like your normal CI on your web repo as you're used to. And then just your web page is configured to fetch the game data files from the game page and then can run it from there. Next step, you might ask, can we actually look at the code? Absolutely. It's all available online. I can also, well, share a screenshot to show the very fundamentals, but I don't really have time to go into the details. So let's look at a higher level abstraction here. What is the actual communication happening between React and the Godot game running on your website? So on the website component, you have a host component. That host component has props of a player name and a power up. It has a callback of like on report score to trigger that score sharing pop up. The host will share these properties through the window object because there needs to be like some back and forth and sharing like the callbacks with the game. The Godot game itself will look on the window object, see, OK, what's my initial player name? What's my initial power up value? And then we'll also set these two functions on player name change and on player power up change on the window. If the props change on your component, these callbacks will be called. And then there's signals on the Godot side. So player name change signal, which will trigger if you want to change the name of the power up. And then the other game components can read the signals. How does it look? Very simple. In the end, you're just registering a callback to say, OK, when the power up change, let's call that function. In this case, this is changing like some of the particle effects slightly. And that's actually most of it from like the technical perspective. So what's working well at the moment? Working well is loading and caching games. It's pretty much feature complete. It's quick. You'd go back to the website, you will get the cached version. So running fine. The communication between React and Godot is working flawlessly. As you did see in the example, it's just going back and forth. Obviously, it could be abstracted slightly more, but about that in a moment. And in general, it's very easy to add new games to your own personal game playing playground. And I would say the developer experience is very nice. I can just create a new game repo. I reuse my workflows, and it's pretty much running on GitHub instantly. So that's all pretty nice. And most important thing, it's a lot of fun because you can use your full game engine, and you can still reuse the games in your personal React website. So take new technology and technology you're used to. What could be improved in the future? Well, garbage collection isn't perfect yet because sometimes the game engine kind of sticks around and is taking up some memory. So I need to look into that. There could be more integrations integrated, like, for example, Twitter SDK to share tweets, or an ad SDK if you actually want to make money with the games, stuff like that. And the message passing between Godot and the game could be abstracted a bit further. You might notice that's all not that fun bit, so I didn't look into that, but I will look into that in the future. Last question, want to try? It's all available on GitHub at github.com. There's different repos you can look in. There's two example games. There's a CI package. There's actually a GSGD bridge package, which is providing the talked about React component. And if you have any further questions, feel free to reach out to me at chaosyogo on Twitter. And thank you very much for listening.