Marrying WASM/WebGL Games with React UI


React is strong at UI development but lags with actual game development. Game engines are great for that but bad at UI. How to combine both?

22 min
21 Jun, 2022


Sign in or register to post your comment.

AI Generated Video Summary

Jonny discusses marrying WebAssembly and WebGL games with React, emphasizing the importance of choosing the right tool for game development. He introduces the Godot game engine as a powerful choice for game development and highlights the limitations of Godot. Jonny demonstrates how to combine React with Godot and showcases the ability to dynamically switch between different games on the same website. He explains the process of exporting a Godot game to the web using WebAssembly. Jonny also discusses the communication between React and Godot games and highlights the benefits and future improvements of using this approach.

1. Introduction to the Speaker

Short description:

Hey everyone, it's Jonny, and today I'm talking about marrying WebAssembly and WebGL games with React, or in some other words, using the right technology for the right job. In my free time, I love to ride bicycles. And the other big hobby I have is games.

Hey everyone, it's Jonny, and today I'm talking about marrying WebAssembly and WebGL games with React, or in some other words, using the right technology for the right job. So, first about me, I work at a social media company as a web engineer, that media company has a blue logo, but it isn't Facebook. In my free time, I love to ride bicycles. That can be helping my brothers with bicycle advertisements, can be a road bike, it can be a mountain bike, as long as it's bicycle related I probably like it. And the other big hobby I have is games.

2. Choosing the Right Tool for Game Development

Short description:

Together with my dad, I have been making multiple board games and developing electronic games. I became an engineer to save time and want to talk about picking the right tool for the job. I started using the Godot game engine and React for time efficiency. React is declarative and great for making a responsive UI. However, it may not be the best choice for games with complex animations and performance is a consideration. Using the correct tooling can greatly speed up game development.

Together with my dad I have been making multiple board games, for example Lost Valley, Nord, Half-Band Heroes, they are usually funded via some crowdfunding campaigns. And since high school, I've also been developing multiple electronic games.

Initially, they were all iOS based, recently I've been branching out a bit, as you might notice from the talk, I talk about making games on the web. Why did I become an engineer? I became an engineer to save time, to save my time and to save other people's time, so they have more time available to play games. And that brings me to the first thing I want to talk about, picking the right tool for the job.

My previous games I largely made with Cocos2D, but Cocos2D is a very code driven game engine, so it's familiar for a developer to just write out the code, but it's not very time efficient because there's no tooling. I started using the Godot game engine, which is enabling me to save a lot of time on the web development side. I love to use React, which is also allowing me to save a lot of time, and I intentionally put it on the board game side here because there's one interesting similarity in board game development and web development with React, and that is being declarative.

If I come up with a new rule to play on the table with my other testers, I just roughly explain them the rule and declare my goal, my intention, what is this rule change supposed to do? Then they will interpret it and use their brain processing power to come up with an interpretation on how to actually move it into the game system, how to execute the rule. On the other hand, if I want to teach a new rule to the computer to play games with, I need to be very precise about everything.

Let's ask the question, what's the quickest way to make a game and why might React be a good choice for that, and why might Godot be a good choice for that? Also, very importantly, what's the most fun way to make a game? Because in the end, this is a talk with no commercial interest. This is just about having fun developing some games.

Okay, so why React? Some people might remember my talk from React Summit Remote Edition in 2020, where I talked about making Diagonal 4. The interesting thing about that game is, first, it's very declarative. I just have a very simple discrete board state. In the end, it's just a bitfield of eight by seven pieces. I can just go from that bitfield, and I can calculate all available moves for every player, and I can just calculate the view very directly because it's just a very simple state and very simple rendering. That maps well to React.

The other great advantage of using React is that it's very easy to make a responsive UI and to have some design systems, making that even easier on the web. Why might React be not the best choice to make a game or modern game on the web? Let's look at this example, Blood Fever. It's a game of a vegetarian vampire defending his coffin against zombies. But the important point is that there might be hundreds of enemies who all are running animations on their own. They don't have a very discrete position. It's more like a continuous state. Some functional Reactive Programming purists might say it's still a state and you can still do like very atomic calculations to come up with possible actions by the player. And then put that in a very big state machine. On the other hand, with animations, it's just not the best thing, I would say, to keep a nice performance running. So just the performance aspect, React might not be the best choice here. And also tooling. Gamedev gets a lot quicker if you use the correct tooling.

3. Introduction to Godot Game Development

Short description:

React is not well equipped for game development. Godot is a free and popular game engine with a powerful editor. It exports to multiple platforms and has an integrated scripting language called Godot Script, which is easy to learn. Godot has a strong community and is a great choice for game development.

And React is really not well equipped there. The ecosystem works. So who might bring good tooling to the table? Godot. Why use Godot for game development? First thing first, it's free software. It's very popular software. If you look at this graph on OSS Insight on the right, this is the Pull Request Creators and you just see Godot becoming the biggest player with most contributors by far in the game engine space. It exports to web, iOS, Android, Windows, Linux, OSX, pretty much all platforms you could need. And it is a very mighty editor. If you look at the screenshot on the left, you see the scene graph. On the right, you see the scene in the editor and the node editor. So you can click simple things and you can move them around, which really makes it a lot quicker than doing all of that just via code. And lastly, it brings an integrated scripting language called Godot Script, which is very Pythonesque and might be simple to pick up for a lot of people. Some other people might still mistype script, but there are solutions for that. But that's a story for another time.

4. Limitations of Godot and Combining React

Short description:

Godot lacks FlexBlock or Grid things, making layouts complicated. In-game menus don't scale well, especially on wider screens. External SDKs may be missing, unlike web development where integration is easy. Web engineers may miss web tooling when using Godot. The solution is combining React with Godot, with some caveats: noncommercial and just a prototype.

Why should you maybe not use Godot? Well, it doesn't have any FlexBlock or Grid things, so layouts might get a bit complicated. Why could that be an issue? Just look at this example from Age of Empires 3. You see a screenshot, the screen was very big, but the menu didn't scale so they do some acting background images. It's just not that great. It can get even worse in game if you have an even wider screen. And where the person would naturally just look in the middle of the screen and only see that. And the menus left and right are very far away, you might need to physically turn your head. So this is just very awful, UI-wise, because the tooling isn't that great, or made for it.

Another reason to not use Godot is there might be some external SDKs missing. One great thing about the web is that it's very easy to just go to npm and integrate some modules into your application. It's a bit more complicated with Godot. Also, on the web you could just easily integrate some authentication SDK from Twitter or from Facebook and then add some sharing functionality, whereas doing that natively in the game engine itself would be a lot more complicated. And lastly, I'm a web engineer most of the time, so I just might miss some web tooling if I just use the game engine. I also want to be up to date with web stuff, so it's a fun way to play around.

So, what's the solution? I mean, the obvious solution is combining React with Godot, but how would we approach that? Well, first, caveat, noncommercial. This talk, as I said, there's no commercial interest. This is just me in my hobby time playing around with games. So, if you want to make money with your games, you might have widely different considerations to think about. The other caveat is just a prototype. I just did start developing this like 2 months ago and was thinking about, hey, how can I combine my two favorite technologies? I came up with this, so I'm showing it at the moment and let's just jump into the example video. So, it's all running on a website. The website is just, in this case, host on localhost, it's on the Playground and there's another game running on the localhost. So there's my game called Kellergewurl und Lauf, which is just a dungeon runner. And you see the connection between the React UI and the game running itself. So I changed my name. Johnny is now popping up in the game itself. What is the game about? You're just running in that dungeon. You can push your adventurer up and down and you need to avoid all the dungeon dwellers and just not touch them. Now, I did touch the developers, so the game is restarting.

5. React UI and Game Export

Short description:

In this demo, I showed how to change the game speed using the React UI and shared the possibility of generating tweets to share scores on social media. I also demonstrated dynamically switching between different games on the same website, showcasing different UI patterns. To get started, create a game in the GoDo engine, like Caligrelden Live, which is a 2D dungeon runner. Exporting the game to the web involves using GoDo's simple export dialog, which generates files including the game.html wrapper, game.js containing engine and game code, game.wasm for WebAssembly, and game.pak for resources. WebAssembly is a binary instruction format that allows applications to compile to a binary format instead of JavaScript.

Now, I did touch the developers, so the game is restarting. I think that wasn't quick enough, so let's use the React UI to change the game speed. And, well, that might have been slightly too quick, so let's slow it down a bit again. And then speed it up again, just to see, like, the connection is instantious, so it's not waiting for the game to reload again to pick up the status changes from the React UI, but it's like a full connection.

And, well, let's talk about a very annoying thing, sharing scores on social media. We might have all been annoyed by the word craze and people spamming our Twitter feeds, but this game also can generate, like, a tweet for you, you are authenticated in the background on Twitter, you can press share and it will be automatically shared to your feed. It's just one possibility of the web integration. But let's look at one other feature, let's switch games dynamically on the same website. Now I'm switching to the remotely hosted version of Caligrelden Live, which has a different scaling mode activated, so now it's just using the design size of the game, so it's just 480 x 320. And lastly, it switched to yet another game, Blood Fever, to just show that there's also some other UI patterns possible, the game we've been using the keyboard and with Blood Fever all is mouse controlled now, and it's all still picking up the context nicely on the web. And that's it for a quick demo combining React with GoDo games on the web.

How do you get there? Well, the first step is, actually have a game in the GoDo engine. Caligrelden Live, as I talked about, is a quick 2D dungeon runner. It's all available on GitHub under GPL license if you want to check it out. So you just create that game in GoDo, you can debug it in GoDo. How do you export it? Well, GoDo has a very simple export dialog. You just say you want to export it to the web in the HTML5 version. You press export, and we'll generate some files. We'll actually generate this doc folder on the left. There's going to be the game.html. This is GoDo's native game wrapper, but that game wrapper really only supports running the game stand alone on the website. It doesn't really have a nice environment to add any further UI on. There's also a file called game.js, which contains some of the game code and some of the engine code, but we're going to look into details for that slightly later. Inside there is a game.wasm file and a game.pak file. The game.wasm file is the actual engine compiled down to WebAssembly and also some of your game code compiled down depending on how you configure the export. The game.pak file is basically all the resources you need for your game, which most of the time are going to be textures or audio files. So Godot exported that, but how can we use that? Let's take a step back initially. Let's quickly talk about WebAssembly and WebGL. That might be relevant for you. WebAssembly, there's this long description you can read if you want to, but roughly speaking it's just the binary instruction format which applications can compile to if they don't want to, for example, compile to JavaScript or some other transpilation, but it's more like a binary format.

6. WebGL Rendering and Local Architecture

Short description:

Godot engine can compile the C++ engine code to WebAssembly, allowing for rendering 3D and 2D graphics with WebGL. WebGL renders into a canvas, eliminating the need for DOM elements. The game.js file handles mapping between JavaScript and WebAssembly code, while the Engine code loads fetch files. Using iframes is not recommended due to potential CORS and messaging issues. The local architecture involves rendering the game in a WebGL context, which is set up by the gsgd host. The infrastructure for publishing games can be simplified by using a Godot project, pushing to a game repo on GitHub, and utilizing the provided workflows in the KS Godot CI package.

And that's very common for native engines to now add another compilation target. So Godot engine can just choose WebAssembly as another compilation target and then can compile the C++ engine code to that. WebGL is a JavaScript API for rendering 3D and 2D graphics. I personally love to do 2D games because it's just easier as a single developer to just pick up some graphics from Open Game Art on the internet for free and use them in their own games. It's just a lot quicker than 3D, but you could also, with the technology presented here, just build a 3D game if you wanted to.

And WebGL also means you're rendering into this canvas. That means there's no further DOM elements for the game itself. It's all a pixel target. It's rendered into the texture, and the texture's rendered in the canvas. Okay, enough for that excuse. Let's jump back into the game.js file. The game.js file considers a large area of two bits. The first bit, which looks very horrifying, that's why I'm not zooming in, is in the end just mapping some of the JavaScript code to some of the WebAssembly code. The connection back and forth works out, but we can totally ignore that because it doesn't really matter at all for embedding our games. There also is some Engine code. That Engine code is loading the fetch files. That's a bit more relevant for us because the Godot wrapper I'm presenting is extracting some of that Engine code.

Now, you might ask, okay, so Godot is providing this like fully. Why can't we just use an iframe? Yes, you could not use an iframe, but you also can't use an iframe because you might have some problem with cores because you're going to run installations where you can't run the code. If you want to, you might run installation where you can't pass messages to the iframe because it's blocked. Also, it's just using iframe, isn't really the React approach, I would say. So, what's our local architecture going to be? Well, first things first, the user visits the website, the websites renders some UI, and that website UI also renders the gsgd host, which is provided in an on-prem package. That gsgd host then goes to some other web host, where you uploaded the game.js, game.pak wasm file which go to exported, and then the host is loading that JavaScript into like the local context. It's picking up the engine definition from the game.js file. It's instantiating that, and it's also providing a WebGL context, and then the engine starts running and is rendering into the WebGL context that the user can interact with.

What is the infrastructure for that? Because it sounds very horrible to set all of that up for all the games you want to publish. Also, again, I think a very convenient and fun solution for that, you just have an embedding website you want to work on, you have a Godot project you want to work on. The Godot project just pushes to a game repo on GitHub. In a YAML file, you can reuse the workflows I'm providing in the KS Godot CI package. These workflows will use a Docker image to compile the game on the CI with the web target, and they will then be published to the game page and you can run like your normal CI on your web repo as you're used to.

7. Communication between React and GoDo Game

Short description:

Your web page fetches the game data files and runs them. The communication between React and the GoDo game involves sharing properties through the window object and using callbacks. Signals on the GoDo side allow for changing the name of the power-up and other game components can read the signals. It's a simple and straightforward process.

And then your web page is configured to fetch the game data files from the game page and then can run it from there.

Next step, you might ask, can we actually look at the code? Absolutely. It's all available online. I can also share a screen shot to show the very fundamentals, but I don't really have time to go into the details. So let's look at a higher-level abstraction here.

What is the actual communication happening between React and the GoDo game running on your website? So on the website component, you have a host component. That host component has props of a player name and a power-up. There's a callback of onReportScore to trigger that score-sharing pop-up. The host will share these properties through the window object, because there needs to be some back and forth in sharing the callbacks with the game. The GoDo game itself will look in the window object, see, okay, what's my initial player name? What's my initial power-up value? Then we'll also set these two functions onPlayerNameChanged and onPlayerPowerUpChanged on the window, If the props change on your component, these callbacks will be called. And then there are signals on the GoDo side, so PlayerNameChanged signal, which will trigger if you want to change the name of the power-up, and then the other game components can read the signals. How does it look? Very simple. In the end, you're just registering a callback to say, okay, when the power-up change, let's call that function. In this case, this is changing some of the particle effects slightly. That's actually most of it from the technical perspective.

8. What's Working Well and Future Improvements

Short description:

Loading and caching games is working well. The communication between React and GoDo is flawless. Adding new games to your personal game playground is easy. The developer experience is nice. It's a lot of fun to use your full game engine and reuse games in your personal React website.

What's working well at the moment? Working well is loading and caching games. It's pretty much feature-complete. It's quick. You'd go back to the website. You would get the cached version. It's all running fine. The communication between React and GoDo is working flawlessly, as you did see in the example. It's just going back and forth. Instantly, it could be abstracted slightly more, but more about that in a moment. In general, it's very easy to add new games to your own personal game playground. I would say the developer experience is very nice. I can just create a new game repo, I reuse my workflows, and it's pretty much running on GitHub instantly. That's all pretty nice. The most important thing, it's a lot of fun, because you can use your full game engine and you can still reuse the games in your personal React website. Take new technology and technology you're used to.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

React Summit Remote Edition 2021React Summit Remote Edition 2021
33 min
Building Better Websites with Remix
Remix is a new web framework from the creators of React Router that helps you build better, faster websites through a solid understanding of web fundamentals. Remix takes care of the heavy lifting like server rendering, code splitting, prefetching, and navigation and leaves you with the fun part: building something awesome!
React Advanced Conference 2022React Advanced Conference 2022
25 min
A Guide to React Rendering Behavior
React is a library for "rendering" UI from components, but many users find themselves confused about how React rendering actually works. What do terms like "rendering", "reconciliation", "Fibers", and "committing" actually mean? When do renders happen? How does Context affect rendering, and how do libraries like Redux cause updates? In this talk, we'll clear up the confusion and provide a solid foundation for understanding when, why, and how React renders. We'll look at: - What "rendering" actually is - How React queues renders and the standard rendering behavior - How keys and component types are used in rendering - Techniques for optimizing render performance - How context usage affects rendering behavior| - How external libraries tie into React rendering
React Advanced Conference 2022React Advanced Conference 2022
30 min
Using useEffect Effectively
Can useEffect affect your codebase negatively? From fetching data to fighting with imperative APIs, side effects are one of the biggest sources of frustration in web app development. And let’s be honest, putting everything in useEffect hooks doesn’t help much. In this talk, we'll demystify the useEffect hook and get a better understanding of when (and when not) to use it, as well as discover how declarative effects can make effect management more maintainable in even the most complex React apps.
React Summit 2022React Summit 2022
20 min
Routing in React 18 and Beyond
Concurrent React and Server Components are changing the way we think about routing, rendering, and fetching in web applications. Next.js recently shared part of its vision to help developers adopt these new React features and take advantage of the benefits they unlock.
In this talk, we’ll explore the past, present and future of routing in front-end applications and discuss how new features in React and Next.js can help us architect more performant and feature-rich applications.
Vue.js London Live 2021Vue.js London Live 2021
8 min
Utilising Rust from Vue with WebAssembly
Rust is a new language for writing high-performance code, that can be compiled to WebAssembly, and run within the browser. In this talk you will be taken through how you can integrate Rust, within a Vue application, in a way that's painless and easy. With examples on how to interact with Rust from JavaScript, and some of the gotchas to be aware of.
React Advanced Conference 2021React Advanced Conference 2021
27 min
(Easier) Interactive Data Visualization in React
If you’re building a dashboard, analytics platform, or any web app where you need to give your users insight into their data, you need beautiful, custom, interactive data visualizations in your React app. But building visualizations hand with a low-level library like D3 can be a huge headache, involving lots of wheel-reinventing. In this talk, we’ll see how data viz development can get so much easier thanks to tools like Plot, a high-level dataviz library for quick
easy charting, and Observable, a reactive dataviz prototyping environment, both from the creator of D3. Through live coding examples we’ll explore how React refs let us delegate DOM manipulation for our data visualizations, and how Observable’s embedding functionality lets us easily repurpose community-built visualizations for our own data
use cases. By the end of this talk we’ll know how to get a beautiful, customized, interactive data visualization into our apps with a fraction of the time

Workshops on related topic

React Advanced Conference 2021React Advanced Conference 2021
132 min
Concurrent Rendering Adventures in React 18
Workshop Free
With the release of React 18 we finally get the long awaited concurrent rendering. But how is that going to affect your application? What are the benefits of concurrent rendering in React? What do you need to do to switch to concurrent rendering when you upgrade to React 18? And what if you don’t want or can’t use concurrent rendering yet?
There are some behavior changes you need to be aware of! In this workshop we will cover all of those subjects and more.
Join me with your laptop in this interactive workshop. You will see how easy it is to switch to concurrent rendering in your React application. You will learn all about concurrent rendering, SuspenseList, the startTransition API and more.
React Summit Remote Edition 2021React Summit Remote Edition 2021
177 min
React Hooks Tips Only the Pros Know
The addition of the hooks API to React was quite a major change. Before hooks most components had to be class based. Now, with hooks, these are often much simpler functional components. Hooks can be really simple to use. Almost deceptively simple. Because there are still plenty of ways you can mess up with hooks. And it often turns out there are many ways where you can improve your components a better understanding of how each React hook can be used.
You will learn all about the pros and cons of the various hooks. You will learn when to use useState() versus useReducer(). We will look at using useContext() efficiently. You will see when to use useLayoutEffect() and when useEffect() is better.

React Advanced Conference 2021React Advanced Conference 2021
174 min
React, TypeScript, and TDD
Workshop Free
ReactJS is wildly popular and thus wildly supported. TypeScript is increasingly popular, and thus increasingly supported.
The two together? Not as much. Given that they both change quickly, it's hard to find accurate learning materials.
React+TypeScript, with JetBrains IDEs? That three-part combination is the topic of this series. We'll show a little about a lot. Meaning, the key steps to getting productive, in the IDE, for React projects using TypeScript. Along the way we'll show test-driven development and emphasize tips-and-tricks in the IDE.

React Summit 2023React Summit 2023
171 min
React Performance Debugging Masterclass
Workshop Free
Ivan’s first attempts at performance debugging were chaotic. He would see a slow interaction, try a random optimization, see that it didn't help, and keep trying other optimizations until he found the right one (or gave up).
Back then, Ivan didn’t know how to use performance devtools well. He would do a recording in Chrome DevTools or React Profiler, poke around it, try clicking random things, and then close it in frustration a few minutes later. Now, Ivan knows exactly where and what to look for. And in this workshop, Ivan will teach you that too.
Here’s how this is going to work. We’ll take a slow app → debug it (using tools like Chrome DevTools, React Profiler, and why-did-you-render) → pinpoint the bottleneck → and then repeat, several times more. We won’t talk about the solutions (in 90% of the cases, it’s just the ol’ regular useMemo() or memo()). But we’ll talk about everything that comes before – and learn how to analyze any React performance problem, step by step.
(Note: This workshop is best suited for engineers who are already familiar with how useMemo() and memo() work – but want to get better at using the performance tools around React. Also, we’ll be covering interaction performance, not load speed, so you won’t hear a word about Lighthouse 🤐)
React Advanced Conference 2021React Advanced Conference 2021
145 min
Web3 Workshop - Building Your First Dapp
Workshop Free
In this workshop, you'll learn how to build your first full stack dapp on the Ethereum blockchain, reading and writing data to the network, and connecting a front end application to the contract you've deployed. By the end of the workshop, you'll understand how to set up a full stack development environment, run a local node, and interact with any smart contract using React, HardHat, and Ethers.js.

React Summit 2023React Summit 2023
152 min
Designing Effective Tests With React Testing Library
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents
- The different kinds of React application tests, and where component tests fit in
- A mental model for thinking about the inputs and outputs of the components you test
- Options for selecting DOM elements to verify and interact with them
- The value of mocks and why they shouldn’t be avoided
- The challenges with asynchrony in RTL tests and how to handle them
- Familiarity with building applications with React
- Basic experience writing automated tests with Jest or another unit testing framework
- You do not need any experience with React Testing Library
- Machine setup: Node LTS, Yarn