Pixi Powerups!

Hi everyone, how's it going? My name is Matt Groves. I'm a front-end developer and also creator of Pixie.js. Today we're going to talk about Pixie power-ups. So basically some of the cool features you can use in Pixie to just make some really nice effects and sort of elevate your content. We're going to do this today via example. So we are going to make this scene using Pixie and I'm going to go through the code and explain what we're using and why to create the different effects. This is basically like a budget Pokemon snap. So you can click on the little fishies, click where you want and it will generate a little Polaroid for you. All right, so let's make this. To start off with, we're going to look at our code. So I'll just tidy all this up. We'll go to index. So here we go. So to start off with, we're going to create a new Pixie renderer. So it's as straightforward as importing a renderer from Pixie. And we go new renderer, width and height. And it's just hard-coded for ease today. So we don't have to worry about resizing stuff. So we create a renderer. Then I just push a little bit of CSS in there that just makes this center aligned and scale nicely. Then we take the renderer's view, which is basically the canvas element that Pixie is going to render into. And we add that to the DOM. Then we create a stage and a scene and we add the scene to the stage and their container, a bit like a development or a node in the DOM tree. Then we use Pixie's ticker object, which is basically a nice, easy way of using request animation frames. So we just add a function to this ticker. And it means it's going to get called 60 FPS. And then what we do every frame is we just go renderer, render my stage. And we end up with this. Well, not quite this. We end up with this, a lovely black screen. So let's go through and build up this scene. And as we talk about, as we hit features, I'll give a little explanation on how they work. So the first thing we want to do is we want to make a background. So if I just go in here, super straightforward. We just create a sprite object, which is like Pixie's most basic thing. It's basic. You go, here's a texture. And then you can just put that texture in your scene and move it around and align it and stuff. So we're not going to do anything. We're going to just, other than just create one using this background PNG. And then we're going to take our scene and we're going to add it to, we're going to add the background to the scene. And let's look at what that looks like. Ta-da! There we go. A lovely, basic background. Next, we're going to add some fishes. So here we, here's our making fishes function. So the first thing we do is we create an empty array of fishes. So this is where we're going to store a reference to all of our little fish. Then we're going to loop through 20 times and do sprite from, and then we're going to use a fish asset. And basically the designer gave me seven, seven fish assets. So this will just basically make a different, use a different sprite asset up to seven. So it go fish one, two, three, four, five, six, seven, and then loop around again. Once we've got a fish sprite, we just set the anchor to be 0.5, 0.5, which just basically means that the center of this sprite object is going to be in the zero rather than top left, which is by, which is the default. Next we set up position of our fishes. So we go, this fish is going to be X coordinate random width. So somewhere randomly in the width and then random height minus 300 so that we don't end up, he doesn't end up going, the fishes don't end up being underground. We then assign a random speed, like plus and minus speed. And then we, we flip the fish depending on its speed. So if it's going that way, we scale it minus one. And if it's going that way, we just keep the scale as one. Then we take our fish, we add it to the scene and we push our fish to our fishes array. Now let's see how, what we get. So we should get this. So every time I refresh, I get some lovely little fishes, just, just moving around. So next I'm going to do is use the ticker again, like we did before for rendering, but we're going to use, use it to move our fish. So all we do is we go, hey, loop all of our fish is a fish and we go right fish at its speed. So it's just going to move in the direction that it was assigned here. And when we get to the edge, we just use a modular, which means that it will just wrap around. So when it gets to that one side, it will wrap around to the other and vice versa. And let's run it now. There we go. We've got some swimming fish. Obviously the cool thing about Pixie is it's quite performance. So you can actually have quite a lot of fish and it's always going to stay nice and smooth. That's a bit too many for us. We'll, we'll go back to 20 I think. All right. So next thing we want to do is add a foreground. So again, just like the background, really straightforward, add it to the same. Oh, there we go. So now we've got background fishes and foreground starting to get somewhere now. Right. First real Pixie power up I want to show you guys is the Pixie rope. So the way this works is, is it's kind of like distorts an image based off an array of points. So I'm going to show you guys what we're going to do with the points. And then I'll show you how that actually maps to a, to a texture. So what I'm doing here is I'm going right. This is the segment length of my space between each point that I want. And I'm creating a points array and I'm looping through and just pushing in points. Now I've got this, uh, show debug points, which we weren't going to, but all it, all it does is returns a little visual of the points that I've just created. And then we can see them being rendered every frame. And then I just sort of center it where we want our real to be. And then I'm just going to add this points view that I've made so that we can see essentially points on the scene. So if I refresh this now, there you go. So these are all points. Not that interesting just yet. Um, the next thing we want to do is we want to animate our points. So if I, if I just show you guys this, uh, what we do on the Y. So we, we had a ticket. So every frame we increment account a little by, by a small amount, and then we loop for our points. And we just basically based on that I position, uh, based on the index and we apply sign, um, maths to it. So what we end up with is the dots just moving up and down now, just to make it extra cool. We also apply the same to the X. We apply cause that just makes it sort of undulate a little bit. If I show you how that works, they get to now you can see there's almost like a, just a slightly more interesting movement there. Obviously moving points is all good. So, you know, how is this useful to us? Well, the magic comes right now. So let's get rid of that point. You, we don't really need that. What we really want is one of Pixies, uh, power ups, the simple rope. So we create a new simple rope and we pass in a texture and we're just going to use that ill texture, which is just a straight deal. And then we pass in the points that we made up here. And then we end up with an ill object and we can just simply add that to the scene and position it just like we did points before. And now you'll see what happens. There we go. You can see that the points are basically distorting the texture and the textures is, is basically mapping itself to the points. I've got a little, uh, screenshot of it here basically. So this is it happening so that this mass is, uh, the triangle is all being calculated for you by Pixie. You just say, here's my points. And then it will, it will let you do these cool effects and you can do tons of really cool stuff with ropes, especially if you introduce real physics. Okay. Let's keep rolling. The next one I want to show you guys is spine support. So Pixie has support for something called spine, which is a really amazing animation tool that instead of key frame based is basically records all the, of the positions and animations in tweens or of, of a designer of an animation made by a designer. And then you can kind of run, run that at runtime in Pixie. So here's an example here and you can see it's just some really cool animations. And the best thing is, is that the designers do all this work. We just load it in. Um, I'm going to show you guys how to do that today. So the first thing you need to do is make sure you install the Pixie spine plugin, which is the runtime. It's not bundled into the main Pixie. Then we just import it here. And then this is our making a spine function. Um, so we create a Pixie loader, which is the way that you can load anything in Pixie really. And then we go loader, add crab, and then we pass in the Jason, which is the format of the spine animation. Then we call loader dot, uh, and then we call load function. And when it's done, it gives a callback with a resource object. Now that resource object has the crab, which we've defined here, and then pass in the spine data to essentially set up the animation. And this is our crab. And it's just like another Pixie object. You can just add that straight to the, straight to the scene. So we get our crab, we set its position, and then we just set the animation to be animation, which is the name, the name of the animation in the animation, which is cool. Then add that to the scene and we should end up with a little crab. Now this is really sweet because all I did was load it in and add it and the designers kind of did all the work and had all the control. Um, and it's a great way to work because it means everyone can do the bit that they're best at. All right, next I want to quickly show you guys the shine effect. So again, really basic. We're just adding a couple of God rays. So new sprite, add to scene, new sprite, add to scene. And then every frame, we're just going to use a cause and sign again on a, on a tick to basically just slide them along across each other a little bit, just to give a bit of interesting movement. Now this gives us, you can just about see them there a bit, a bit sort of dusty, but they're right there in there in the animation. So what we can do is use one of Pixies blend modes. Rather than rendering a normal style, we can use an add blend mode, which basically is an additive as it adds pixels as it draws rather than does the normal kind of alpha blend. There's a few different blend modes available in Pixie. A lot of them only work on canvas, but some of them, you know, quite a lot of the useful ones like overlay and multiply, they all work in WebGL too. So we're going to just use add and I'll show you what the difference that can make. There we go. It just looks much more like light and add always is always a great one to use if you want to make something, you know, like light scene up. Okay, moving on. The next thing I want to show you guys is how to use Pixie filters. Okay, so let's go in and check out filters. All right, so we're passing in our scene again, just like before. And this time, we're going to create a new, we're going to import the filters object from Pixie, and we're going to create a new blur filter. Then for our scene, we just, every object in Pixie has a filters object that you can assign filters to. So here we just go, filters is this blur. And that's basically all it takes to create a filter and apply it to anything you want in Pixie. So if I just refresh this now, there we go, we've got a version that makes it very difficult to see what's going on, but proves the point. So that's a useful filter. But here, I want to show you guys a slightly more advanced one. So what we're going to do here is create the water ripple effect using what's called a displacement filter, which is another standard filter that is definitely one of my favorites. And we use it wherever we can, but water basically shows off the best. So what we do is we create a water displacement, we load up a sprite from our water displacement JPEG. And then we set the base stitch to repeat. And then we set scale to be six, so it's quite big and takes over the screen. And then we add that displacement sprite to the scene. So I'll show you guys what that looks like. There we go. So this is the displacement sprite. And the idea is that anything in the red channel shifts everything pixels X wise, and anything in the green channel shifts pixels on the Y. So yeah, not that interesting just yet. But if we then create a new displacement filter, and we pass that sprite in, and then we set ourselves a strength of 100, we can then apply that filter just like we did the blur. There we go, just like this. Now if I refresh it, now you can see we're no longer rendering it, but we're actually using it as a filter input. So you can see everything's kind of a little bit more ripply. Now to really sort of show you that to the extreme. There we go. That's what it can look like if you if you make it a bit nuts, but it kind of maybe doesn't look quite as good now. So let's put it back down to 1000. And then the other cool thing you can do is now that we've got the sprite in our scene, I can just move it around. And as it moves, it will displace pixels in a different way. So here we're going to basically go here's our here's our displacement sprite, and then every frame just move it up two pixels. And this is what happens when you do that. So now you can see it just looks like we've got this really lovely water ripply effect. Now you can do so much more with displacement filter you can I mean, you just change that texture to be something else and it will look completely different. You change the strength, you can change the x strength, the y strength, it's really powerful filter. The other thing I'll just quickly show you guys is it's really easy to add multiple filters. So if you wanted to blur and a displacement filter, you just go like that and you just pass in an array of multiple ones. If you wanted to blur it twice, you could even do that. So if I show you that now, there you go, that's like double blurred and with a displacement filter. We can even displace it again at the end. And it's here so you can stack as many fields as you want. Obviously, the more you add can have performance and implement implementable performance implications. But generally speaking, you know, you should be all good. Alright, moving on. So this is just a really simple one. This is making adding a vignette. So a vignette graphic around which just frames the content. So if I show you this now, just that black edges just makes it look a bit more nice. Designers, our designers love to give me a vignette. Right. So the next one I want to show you guys is the image capture stuff. So basically making this into the budget Pokemon Go. So Pokemon snap that we talked about. So let's go in here and have a look around, have a look around and see what's going on. So the make camera function essentially takes our scene and then image container. So once we make our little Polaroids, that's just the container we're going to add it into. But this is the scene that we're going to snapshot. So what we do is we create first we make our scene interactive. Once you've done that, you basically get the same events that you might get in there in the DOM. So pointed out pointer events, touch events, mouse events, all that. So we go right scene we want to be interactive. And then we go okay, I'm going to listen for the pointer down. And whenever we do this function will get pulled. Right, so here's our here's our magic. Basically, we create something called a render texture. So we just go render texture dot create. And we give it a width and a height. So that's given us a basically a 512 by 512 empty texture with the ability to us to add right pixels to and do whatever we want and draw whatever we want to that. But yeah, it starts off blank. And then the other thing we do is create a matrix which is going to be used for when we say oh, we want to render the scene to this render texture, we want to say where and that can be represented as a matrix. And so you don't have to fully I guess fully understand all this stuff. But it's you know, if you do awesome. So the first thing we do is we set the TX and TY which is like the translation of the matrix. So we go when we render the scene, I want to translate it by this is actually the global mouse x and global mouse y coordinate. So it means that I want this to be this this maths here basically says wherever the user clicks, I want to render the scene with that point being the middle of the scene. This just caps that value. So it means if we're too far off and part half of this is half of the capture is going to be off screen, then we just make sure that it doesn't go over that. Then we've created this. This is the magic line here, right? So we go render dot render, which we were doing before we were doing it here, you know, render dot render stage, we do that every frame. And make camera when you click we go render dot render, and then we pass in the scene, then we pass in the render texture, which is basically this is it says this is what I want to render it to. So I'm in this scene to this to this texture, rather, if this is empty, then it renders to the screen. Otherwise, it renders to this render texture. This is the clear property do we want to clear it true or false, not really important here. And then the matrix so like, what, how should you know what part should I draw and how should I transform that. So once we've got to here, we've got we've now got a internal texture that we can then use and attach to sprites and stuff and then just use as if it was another texture that we just loaded in. Alright, cool. So. So moving along, this is just a picture container. So this is how we can just represent this image and make it look like the little Polaroid. So we create a container and then I'll create the background, which is like the Polaroid frame. So it's just a white with a little black, white rectangle with a black rectangle behind it as a shadow. This is used using a pixie graphics object, which is like a really simple drawing API. So I just super quickly walk through what we're doing here. So again, begin fill zero, black of an alpha of 0.3, then draw this rectangle, then that's just an x, y, width and height. Then we begin another fill with a white fill and we draw the same rectangle again, but slightly offset. So that's how we're going to that's our Polaroid drawing. Then we create a new sprite and we pass in our render texture as the sprites main texture. And we set the position and then we add the background and we add the sprites of the picture. So that's our Polaroid. Then we add the picture to the image container. And then we just do some alignment, which isn't so important here. Now, if I show you guys this now, now whenever I click, you can see I can capture a lovely Pokemon Go style, sorry, Pokemon Snap style. And see if I can get this crab when he jumps. Come on, crab, jump. There we go. Perfect. So there you go. That's how you can use render textures to, you know, really do some cool stuff like taking pictures of things. So the final thing I want to show you guys is, right, it's all good having another image within your scene. But what if you want to download that image and actually just save it as a legit real image. To do that, this is the code you need. And I'm just going to quickly walk you through that. So basically, we have our renderer and it has a module on it called extract. So we just go renderer.extract. And there's a few different ways in which you can extract information. You can extract it as a canvas as a base 64 encoded string, or even just a regular HTML image. But for us to be able to download it, the best thing for us to extract as is a canvas. So we go hey, renderer, extract a canvas from me from this render texture. And that gives me back a regular canvas filled with the pixels from this render texture. Now I've got this, I can call the toBlob function, which basically is converting it to a blob that's a JPEG. And this is the quality of the JPEG. And it's got a callback. And once that's done, I get given back my blob. Then I use a URL, create object URL, which gives me a URL that can basically be downloaded. I create a link element, I set the href to be this image URL, I just set the download name, and then I append the link to a fake click, and then I remove the link. And that's it, that will download an image. So I'm just going to show you that now. So there's me creating my images. And you can see my Chrome is starting to go a bit nuts downloading all these images for me. And I'll just show you guys those in motion. So here we go, you can see, there you go, here's all my C shots that I can now share with all my friends and users, desktop background, or whatever crazy thing is you want to do. All right, sweet. So that brings us to the end of the talk. If you want to have a play with this, with the scene, you can go here and you can just have a little fiddle around, take some pictures of the crab and the eel. But even cooler, you can just get this code, it's all open source, all the assets are open source. So you can get this code, download it, have a play, kind of get to understand it a little bit better. I know 20 minutes is a bit brief to maybe jam all that stuff in. So here you can kind of have a proper look and take take your time. Cool. That's it. Yeah, follow me, doormat23 on Twitter, if you want to be buddies. But yeah, thank you very much for taking the time to listen to me. Cheers. Yeah. Thanks for joining. So the question was, which of these would you say is the most important developer trait outside of coding? And the winner, well, by far is communication with 49%, followed with 39% for positive attitude. Matt, what do you think about this? Do you agree? Yeah, I think that's, I think that sounds about right. I think I agree. Intelligence is definitely at the bottom because you can do all of this stuff without being naturally smart. I think grit is maybe slightly underrepresented because I think grit is like the substitute for intelligence. I don't consider myself particularly smart, but I do consider myself, if I've got a problem, I'll keep chipping away and I know eventually I can solve it, which kind of means I don't have to make my brain bigger, which is quite difficult to do. So I think, yeah, grit is definitely important, but yeah, communication is above all, right, because we write code for humans, we talk with humans, we make things for humans. So everything needs to be for other people first and really the computer last. Yeah. Yeah. I actually wouldn't be surprised if positive attitude would be above because if you're going to be in a team, unless you're working alone, of course, but if you're going to be working in a team, being positive and being nice to work with is even more important than your coding skills, I think. You can be the best coder, the fastest, the most secure, most accessible code writing person, but if you're not nice to be working with, then no one really wants you in the team, right? Yeah, totally true, man. And it even scales to projects and teams in general, right? It's like you can make the same end result, but the journey there can be horrible and painful or lovely and respectful and you can get the same outcome, but the team won't want to work with you again. They won't want to, you know, so it's yeah, definitely. Yeah, I guess they're all kind of, apart from intelligence, they're all kind of important. Yeah. Well, but your question was, what is the most important? And we got down to that. Yeah. So everyone that's watching, I want to remind you that Matt's open for questions. So go to the Discord channel community track Q&A if you have any questions for Matt about his talk. While people are typing, Matt, I was curious also about your journey into becoming a developer. Can you tell us a little bit about that? Yeah, definitely. So I started to learn to code when I was quite young. My dad's a programmer and we had an Atari ST. So he sort of showed me how to do really basic stuff, you know, like go, you know, endless loops and all that kind of stuff. And I did kind of end up writing a tiny little basic game where like it just moved the pixel across the screen and you press the button and it shot it. So that was my first stab at coding. And then I kind of put that away for years. I decided I wanted to be into art and drawing and all that kind of stuff. So you built Space Invaders. It was like rubbish Space Invaders. Yeah. Well, pretty cool as a kid, right? So yeah, yeah. I was I was super happy with it. And so yeah, and then I kind of took a designer path, went to study, sort of studied, always studied like computers and design it the same sort of thing. And then when I got to the industry, I sort of started off as a junior designer because that was what I thought I was best at. But as it turned out, there was this sweet spot that there's kind of like creative coding, which is kind of like you can do visual things, but you do it all through code. And so I started kind of getting into that side of things. And that's when Flash was around and Flash was a really big, big thing. So I kind of, you know, that whole time, you know, where people were just making crazy fun things, I kind of was just, you know, wanted to be part of that. And then, yeah, working, working in Flash until until its demise, and then switched gears to mobile games, was iOS developer for a little bit. And then, and then JavaScript got really good all of a sudden with WebGL and all that and sort of switched back. And haven't looked back since it seems to be the best platform out there for reaching people. Yeah, yeah, it's the web is the most well, JavaScript is the most accessible language, I guess, on the most devices. So that's true. That's funny. It sounds really similar, because I also actually, I wanted to be an animator. So I was animating in Flash. And at a certain point, there were some things that I wanted to do that were repetitive. So I thought you could do this with code. There's this action script thingy. So yeah, go to and play frame this and that. Yeah. And at a certain point, these animations, oh, the more and more I animated, the more I wanted to automate and write code for. And that's actually how I got into development. Yeah, I really knew that I wanted to do this. Yeah, my Yeah, I'm not too dissimilar. It definitely started off like it was Flash. It was all about just animating in the timeline, go to and stop, go to and play. And then yeah, like then you realize you can code things. And all of a sudden, you kind of do this. And then before you know it, you're writing more code than you are animating. And then, and then you've unlocked, you know, the ability to do like games and really crazy stuff. So yeah, it's kind of, we need that back in the world. Not Flash, but some sort of creative tool that covers that. Yeah, a WYSIWYG JavaScript editor. Well, we have Dreamweaver. I don't know if that's still a thing. Yeah, I think the closest to the kind of gaming world is probably things like Unity. I know that there's some really cool engines out there as well. But kind of like Play Canvas, which is kind of like the web friendly version of what Unity is trying to do. So I think there is a space for game makers. That does exist. But yeah, Unity to web is not mature yet. So it means that if you want to make web games, that's probably not the most viable option because, you know, web is all about being teeny tiny, very small, and getting users in as quickly as possible. And I think these kind of transpiled things right now, they're just a bit too cumbersome for the web. But that's just now, in a year, that's all going to change. Yeah, I mean, tomorrow, something could come out that makes it possible to port a Unity game to JavaScript. So I mean, you can do it like there is an export, like they do publish out to WebGL. And it's quite phenomenal. What you get is just, yeah, it's just high. It's quite demanding. The computer you would need is more demanding than the computer you'd need to run the native one. But there's loads of cool stuff in the web coming through, right? Like we've got WebGPU coming through soon, which will be a whole new way of talking to the GPU, which is going to, you know, the last thing I read, it was going to be up to two to eight times faster than WebGL. Once it's, you know, if you remake the same thing, so even things like Pixie will be able to remake it and have a WebGPU render, which means that we'll be able to just get everything running even faster through this API. So I'm super excited about that one. That sounds pretty cool. Matt, I wanted to thank you for joining us here today. And yeah, if anyone wants to ask any questions, I think you have a speaker room on Spatial Chat where you're going to go now, right? Cool. Yes, I'll be there. Yeah. All right. With that, once again, thank you for joining us and hope to see you again soon.