Games Are Smarter Than Us
AI Generated Video Summary
2. Building Game Elements and Using Game Engines
To build a game, you need to put your game elements in the code and use the Canvas to draw them. You also need to listen for controls and implement the game logic. All of this happens in one frame of the game, which is controlled by the game loop. Drawing on the Canvas can be challenging, so sprites are often used to simplify the process. If building a game from scratch seems daunting, game engines like Phaser, Pixy.js, and GDevelop can provide a framework to focus on graphics and logic. Another option is the bitmello.com framework.
And then you have to put your game elements. Since you don't use React and don't use state management, you just put elements, you just put objects in your code, and for example, you have the first player, and you have its coordinates, and is it running or not, is it healthy or not, and then you have the enemy, for some reason it's John McLean, but Fry fighting John McLean, and you have its coordinates and the velocity, and these are the things that represent your elements, and then the Canvas, you will use the Canvas to draw from these objects.
You have to listen for controls, right, so you have the keys, and then you listen to event listeners on the window key down or key up, and once you listen to those events, you register the key. You say, okay, if the key was pressed, then I put it as true, then I put it as false, and this is the code that does that, right? You say the key code is left, then you put keys.left is pressed, and then you have in the keys object, you have the list of all the keys that are pressed.
Then you have to implement the game logic. So, the game logic is actually where the game takes place. So, for example, you say, okay, if the keys.left was pressed, then I will increase the Batman velocity X. If it's down, I'll increase the Batman velocity Y, and then you have to say, okay, to detect for edges, if the Batman.y is bigger than the screen height, then I have to put it in the screen height, and then you say, okay, if Batman and Superman are the same X, then you'll reduce Batman's lives. Otherwise, if he takes the heart, you increase his lives, and if it's smarter, you break. And this, all this is to you have to make sure that the code follows the logic of the game that you want to do.
Now, bear in mind, all of those things are happening in one frame of the game. So that brings us to the game loop. You have to run a loop using request animation frame, and you have to do with all the magic of the game inside the loop. And every animation frame, you run all the logic of the game. So, you basically update everything according to the keys that were pressed, like we saw. And then you run the actual game logic. You detect collisions, you detect edges, everything. And then in the end, when you have the updated objects, you draw them on Canvas, and then you run the next frame, and the next frame, and the next frame.
Now, we talked about drawing on Canvas, but it's not that easy. You can see here, for example, how to draw this pixel art character on the canvas. And you see that it's very, very, very, very, long to do it in code. We usually will use sprites. Sprites are, as you can see here, you have all the modes of the character in one image, compressed into one image, so you can just use the part of the image that you want according to the position and the velocity of the character.
3. BitMellow: Exporting Logic and Using React
So let's see. When we go inside bitmello, we see this kind of screen. You can see here on the left, we have the tile editor. The tile editor basically lets us edit the tiles that are going to be used in the game by us. So we can edit the actor. We can draw here something else if we want, if we want to add another character. We'll see later in the talk how we're going to implement this. And you have the tile map editor, and that basically helps you edit the map of the game. You have the sound editor, which is very nice. You can add sounds to the game. And the most important part is the code. Don't worry if it's too small here. We're going to go over it later in greater details. And you can export your project, which is nice. In the tile editor, you can see here on the right some tutorials. So if you want to plan animations, you can see here how to plan the animation. If you want to draw clouds, you have some explanations how it's the best way to draw the clouds, how they should flow. So that's why I really like BitMellow. It's exporting an HTML. It's not something we can use, it's exporting a single HTML that basically includes all the data of the game. And we don't want to use that. And for that, for those of you who wondered how any of that is connected to React. So we'll use Npx create React app to the game, and that's the first and the last appearance of React in this talk. So we're going to use a game, sorry, we use the create React app and we're going to build ourselves like a small React app, and we're going to break the HTML into the parts. So we're gonna import the BitMellow engine. We're gonna take all the game logic that we wrote and put it in game.js, and we're gonna import from the project data JSON. And in the React, in the game wrapper, we're just gonna init the game logic. This is the JSON. And the nice thing about BitMellow is that it lets you export only the JSON.
4. Building Games with BitMellow
You can edit and export your game in BitMellow, and easily build games using the BitMellow engine. The engine handles drawing the map, targets, and player, as well as updating the player's coordinates and checking for target grabs. It also handles animating the player's movements based on the frame count. With BitMellow, game development becomes simple and enjoyable.
So you can keep editing in BitMellow and exporting and overriding only the JSON. Okay, so let's see what we have. So we have here this Qt game. So you can easily write a game like that, where you just go and collect mushrooms. And you can animate the text and you can decide which mushroom appears where. And you can see here the text that tells you how many mushrooms you've collected. And basically what it does is that you can see that engine on update, that's the loop that we discussed. So it draws the map and then draws the targets. So for each target, for each mushroom, if the target wasn't grabbed already, it draws the tile. And it's really easy, just screen dot draw tile, that's all we need. And then it updates the player. So what does it mean to update the player? So you just taking the coordinate of the player, you check four keys like we saw. And we saw and then you update the coordinates. And then you calculate the distance to the target. And if the distance is small enough, you say, OK, the target was grabbed. And the last thing is just drawing the player. And like you saw in the tiles, you have three tiles for the player because it can go when it walks, it lifts one leg and then lifts the other leg. So you say, OK, you count the frame since it started walking and then you decide which tile to drop. So it's really nice. Yeah, so it's safe to say that we leveled up and now we can build, easily build games using the Bitmallow engine.
5. Using AI to Make Computers Play Games
Now, let's explore the concept of using AI to make the computer play games. AI can analyze parameters and make decisions on what actions to take. Unlike coding rules for every step and state, AI can infer rules from data. However, in computer games, we don't have all the inputs and results, so reinforcement learning is used. This involves teaching the computer to learn from rewards and punishments instead of labeled data.
And now it's time to do something a little bit more interesting. It's time to visit the Mountain of Wisdom. And what does it mean, the Mountain of Wisdom? Well, building our own game wasn't the end goal for us, right? We wanted to make the computer play it. And the idea is that in the last years that you can see a lot of papers, a lot of work about AI playing games. And you can see here a StarCraft by DeepMind some from the last years. And it's not as complicated as it seems, but it requires a lot of training. And we'll see in a second how does it work behind the scenes.
So you can see here an AI playing Super Mario. And you can see here what happens behind the scenes. So you can see what happens behind the scenes. Well, the AI looks at all of the parameters and then decides what to do, what kind of action to do. Okay. So the thing about using AI instead of coding the rules, is that if you want to play a game by coding the rules, you have to code and to tell the computer what to do in every step, in every state, meaning if the player.x larger than enemy.x and the player is enough health then move is run from the enemy. And when you use AI, you just try to infer the rules from the data. So you tell the computer, okay, this is the input. This is the result. This is the input. This is the result. This is the input. This is the result. Just train on that. And then I'll give you a new input and you will tell me what's a new result. The thing is that with computer games, we don't have all the inputs and we don't have all the results in So for example, in StarCraft, we can say, okay, in this frame, do that. In this frame, do that. In this frame, do that. Because we don't know that. So we have reinforcement learning. Reinforcement learning, meaning that we teach the model, we teach the computer to learn from rewards and punishments instead of results. Instead of labeling the data. What does it mean? Let's say we have an agent and we have the environment.
6. Observing State and Evaluating Actions
The agent observes the state, runs processing, and selects the best action based on immediate and expected rewards. The environment returns a reward based on the action, which can be zero, positive, or negative. Actions are evaluated by comparing their outcomes and updating a table of state-action values. Reinforcement learning involves trying actions, receiving rewards, and learning their effectiveness.
So the agent wants to play the environment. So first of all, our agent needs to observe the state. And the state usually means what happens right now. How can I quantify what I see right now in the game? For example, for super Mario, it can be the horizontal, the vertical axis, how many enemies do you see, how many lives, how many coins. That can be the state.
And the agent observes the state, it runs some sort of processing and then it invokes an action and it sends the action to the environment and it doesn't know anything about the environment. It just knows that it needs to send the action there. So the environment gets the action and in return it returns a reward. And the reward can be zero if nothing happens, and it can be positive, it can be negative. It depends on the outcomes of the action and the environment also changes the state. So it sets a new state.
So now the question is, how good is every action? So when an agent sees a state, it needs to know how good, it needs to look at all the actions and select the best one. So how do you measure the best one? How good is the action? Well, you calculate the immediate reward that you get from the action. And since it brings you to a new state, you also calculate the expected reward from this new state. So if it brings me to a better state, this action is better than some action that brings me to a worst state. So basically we just hold some sort of a table, and in this table for every state, for example, now we're in state two. We look at the actions and we see that the best one here is up, right, because it has a value of 11. So we select up. And now if we select up, we get some sort of reward, so we get a state two, and then it moves us to another state because we were in state two and then we selected up, so now we're in state three. In state three we select the best action here, which is right, but we also update the value of the up action from the previous state because now we know it gets us to a state where the value is 10. So now I updated it. So basically, reinforcement learning is just try and fail and try and get a reward and try and fail and try and get reward and try to learn how good are the actions when you do that.
7. Implementing Flappy Bird and Reinforce.js
Okay. And here's an example of a flappy bird, something that tries to play flappy bird, if you know, and basically that's what it does, right? It saves in the table the state and whether to flap or to do nothing. And it takes a lot of iteration. It takes something like 25 minutes until it learns to play it completely.
And if you have a very complex state where you can't put it in the table, you just put a neural network and that's a whole bigger world to discuss, a neural network that gets the state and then outputs the action. That's what we're going to use. And for more complex games, you don't even use a state, you just feed the frames. This is the frame of the game, just decide what to do. This is the frame. The actual pixels, not the state. That's what autonomous cars are doing.
Okay. But we're gonna use something that's called Reinforce.js. And it's really easy to use. You just define the states that you expect and the actions that you want. And then you initialize an agent. You act on the state. You tell the agent, act on the state. And then the agent returns the action that you need to do. You try to execute the action and get the reward.
8. Teaching the Agent to Learn
To make the agent learn, we initialize it, act on the state, get the reward, and learn from it. The AI.js code is straightforward: it gets the actor and target, calculates the distance between them, acts on their coordinates, invokes the actions, gets the new coordinates, and determines the reward based on the distance to the target. We can now watch the computer play the game, earning positive rewards for getting closer to the mushroom and negative rewards for moving away from it.
And then you tell the agent to learn from the reward. And it's that easy. That's it. You just initialize the agent. You act on the state. You get the reward. You learn from the reward. And then in the loop, you do it all over again.
So, that's what we're going to use. We're going to change Game.js to expose our actors, the actor and the target. And to allow us to fire action. And we have here the AI.js code. I'll share the code later. But basically, it's really easy. It just basically says okay, get the actor and the target. Get the distance between them. Act on their coordinates. Invoke the actions. Invoke the action. Get the new coordinates. And then get the reward. And the reward is did you get closer to the target. Then we'll get positive reward. If we got further from the target, we get negative reward. And we learn from this reward.
So, now, we can just start playing give the computer to play for us. So, you can see it on the left. And you can see our agent playing and exploring. And it gets positive reward if it gets closer to the mushroom. And it gets negative reward if it gets further from the mushroom.
9. Computer Learning and Model Optimization
The computer learns to get closer to the mushroom and improves over time. The rewards are negative as it moves further from the next mushroom or cake. The model can be saved and loaded for better performance. When the agent reaches higher levels, it becomes very good.
You can see it gets stuck but quickly it gets to the mushroom. And it learns to get closer to the mushroom. And that's the computer playing. And it's learning and learning. And you can see the rewards are negative because every time it takes a mushroom or a cake, it gets further from the next one. So the cumulative reward are negative. And since everything is modeled in the network, we can actually save this model and then load it. So if I load the model or I load a previously trained model, my agent should be better. My agent should be, it should use all the knowledge that it already learned in previous iterations in order to play. And you can, I'll share this website later and you can just play it and you can see that when it gets to 30, 40, 50, it gets really, really good. So cool.
10. Exploring AI and Modifying Rewards
Now we have AI, which is very good. We can add other tiles and change game logic without modifying the AI code. Rick can collect planets in a space-themed game. By changing the reward, Rick learns to get further from Morty. Rewards for getting closer to something and getting further from Morty can be added.
Now we have AI, which is very good. And now we can start a little bit to explore with that. Yeah, I think we deserve a level up for that.
Okay, so now we get to the final chapter, the river of opportunities. And since we have AI, and the AI is not connected at all to the game, we can just add other tiles, add different game logic, and the AI will still be able to play this game. So let's say I draw a lot of other tiles now, I draw Rick and Morty and I draw some stars and I want my game to be in space, so I put my game in space. I don't change at all the AI code, and I just start. And I let my Rick try to collect the planets. So you can see it collect the planets. I loaded the previously trained model from the mushroom game, but it doesn't matter because all I changed was the graphics and some of the logic of the planets. And you can see that Rick is still a little bit confused, but it won't take long until it takes Mars as well, and then cake. And so that's really nice. And if you let it run, it gets really, really good.
We can also add another player and change the reward. So instead of rewarding for getting closer to something, we can reward it for getting further from something. So let's say we have Rick and Morty, and we have Morty running around here, and we will input Rick's coordinates and Morty's coordinates and we'll change the reward. So instead of rewarding for getting closer, we'll reward for getting further. So we'll add a minus here. And then we can see that Rick's trying to escape Morty, and I can't load anything here because the model that I trained earlier was about getting closer. So now we need to let Rick learn on its own to get further from Morty. And you can see that whenever Morty gets closer, Rick gets further away and eventually gets really, really good at that. And the rewards are getting higher and higher. Well, I think Morty is picking Rick here, so we put it in the corner.
The last thing that you can do is try to add the reward. So the reward for getting closer to something and reward for getting further from Morty. And let's see how it works. So you see the reward here is minus one for getting closer to Morty and plus one getting closer to the target. And here, Rick gets a little bit more confused because he needs to move in the space in that kind of way. For example, this situation where Morty is very close to the target, he doesn't know what to do. And you can see the rewards are getting negative very fast.
11. The Journey of Game Development
We went through different stages in our game development journey, from the village of boredom to the pit of despair, and then to the valley of pixels where we learned game building. We reached the middle of efficiency and the mountain of wisdom, where we added AI to our games. Finally, we arrived at the river of opportunities, realizing the endless possibilities with our model. So, don't just dream, take action and implement your ideas. You're awesome!
But again, if you let it run for a few minutes, it gets really, really good at that. So what did we have? We have the village of boredom, which led us to the pit of despair. And then we had the valley of pixels where we learned how to build games. The middle of efficiency, the mountain of wisdom where we learned how to add AI to the games. And then the river of opportunities where we saw that once we have this model, we can do everything, anything with it.
So don't let your dreams be dreams. Just do it. Just do it. If you think about something, just read about it and try to implement it. And that's it. You're awesome. I was Liad. Here you can see I have the repository and the website to play with, and I'll share the presentation.