JS Nation

JSNation 2022

In this talk I will show Makepad, a new UI stack that uses Rust, Wasm, and WebGL. Unlike other UI stacks, which use a hybrid approach, all rendering in Makepad takes place on the GPU. This allows for highly polished and visually impressive applications that have not been possible on the web so far. Because Makepad uses Rust, applications run both natively and on the Web via wasm. Makepad applications can be very small, on the order of just a few hundred kilobytes for wasm, to a few megabytes with native. Our goal is to develop Makepad into the UI stack of choice for lightweight and performant cross-platform applications. We intend to ship with our own design application and IDE. by <a href="https://www.froala.com/wysiwyg-editor?pb=1"> </a>

Makepad - Leveraging Rust + Wasm + WebGL to Build Amazing Cross-platform Applications

All right, welcome to my talk. I'm going to talk about Makepad. It's a new way to build UI for web and native using a WebAssembly and Rust. So I want to start this journey long, long ago, maybe before some of you were still in primary school even. I founded Cloud9 IDE. It's a company to build a web IDE. We had a HTML-based code editor called Ace. It was one of the two at the time. And so I was in the middle of HTML world building what was considered a complex application, an IDE. Everyone was making web pages, some web apps, but the IDE with the code editor was considered a complex web application. So at the time, I was working on the editor. And I remember we were working on code folding. And I wanted the code folding to animate. It's a very simple thing. You know, computers are fast. They can animate it. But no matter what I tried, I couldn't get it done with HTML and CSS. Whatever we did, it just was choppy and it sucked. So at the time, WebGL was just released. And I started investigating using WebGL to render UI, render editors. And I realized that this was a huge task ahead of me to reinvent the UI in WebGL. So I left Cloud9. And I started on the journey of redoing the entire render stack of the browser in WebGL and JavaScript. And I spent six years on that. I probably built as many, six or seven, complete file new iterations of trying to do this. And I want to show you the result of those six years here. And that is MakePad JavaScript. This is MakePad JavaScript. It is a complete do-over of the render stack in WebGL. And you can do, let's see, you can live code. All of this is live codable. So let's see if I can change the color of the text here. I can also, let's see, I have a bit of feedback on the audio. Yeah. So it was really fast. It was really, really fast. This is 50,000 interactive circles in a browser. It was using multi-threading. So this application running here was running in a worker thread and sent over the display list to the main browser thread. So you could have any number of applications running at the same time as long as you had workers. And you could do very, very cool interactive things. This all runs also on an iPhone 6, I think, still, if you load it up in Safari. And at some point, I even managed to run itself in itself. And this is great. This is all fine. But it had a big problem. It had a big problem. At the time in Chrome, when I was working in the application for more than 30 minutes, Chrome would just crash. It would just crash, and I didn't know why. And having dealt with browser bugs before, sometimes they take years. Sometimes they never get resolved, especially when you're like, yeah, I live code in Chrome for half an hour, and I hot reload a worker. That's a bug. They go, OK, you're probably the only person in the world that does that. So good luck with that. So that was very depressing. And hold on. So it wasn't stable. That was really a problem, because I wanted to use it myself. If I can't use my own developer tooling for my own stuff, then the developer tooling sucks. However, there was a real problem, because maybe I could have gotten this fixed. I could have fixed it myself or lobbied forever to get someone else to do it. But the real problem is that you don't have CPU power left if you do all of this in JavaScript. A worker thread just caps at 100% if you use the editor, the syntax highlighting. All of that stuff is just too heavy to do in real time with JavaScript. And that meant that I was looking ahead. I see AR, VR user interfaces. I see high frame rate screens. The latest max are 120 hertz. So you don't have that 16 milliseconds anymore. You're down to eight or lower. So it's a dead end. It's a dead end. The JavaScript was wrong. Essentially, it's the wrong technology to try to do this, especially because I wanted to build complex applications like IDEs, design tools, 3D modelers. These kind of tools I wanted to build on the web. But if you try to do it in JavaScript, just the data structures that you need to build anything complicated will drag down or hiccup your application to no end. It's sort of a limited domain. And dynamic typing also doesn't scale. TypeScript sort of alleviates that. But with JavaScript, if you scale your code base and you try to build on things that you did before, at some point it's going to start to fall apart. So this was six years in. Was this the end? Did my life have no purpose? It's a bit much to sort of throw away your work at this time, even though it looked great. It looked nice. So yeah. And then Rust came along. Rust is a relatively new programming language developed out of Mozilla. It's a borrow check language. It doesn't have a GC. This is a very fundamental difference, which is also why it takes time to learn. You can't just randomly create objects and magically have them go away. This is also why it's fast. It doesn't have GC hiccups. It's compiled native, and it compiles to WebAssembly. And it's both high and low level, which is different because JavaScript is kind of a mid level language. It's not very high level, but it's also not low level. And that is very limiting because sometimes you want really high level constructs, like high performance iterators or kind of trade systems. And sometimes you want low level because you need the performance. So I had a new chance at trying this out with a new technology. And I joined forces with Eddie, who is an ex-Mozilla developer who was there when Rust got invented. So he knew Rust, and he could teach it to me as well. And Sebastian, who is one of the most brilliant UI designers I know. And because if you're building UI, you better try to make it look good. And I just play games. So make that Rust. This is a complete reimagining. So first off, this is make that JavaScript, and here's make that Rust. It's the same idea. It has a code editor. And now I could finally do animated code folding. This is a feature that shouldn't be hard. But considering that you're clapping, people do consider it hard. Because this is what I wanted. I wanted to have an overview of my code that just folds back. And I keep my spatial memory. I don't want it to code to fold to nothing. I want to see that there's body of work there. And you can just skip your cursor around. And it also still has the live editing. Like here's the live thing going on with the changing code. I'll tell a little bit how that works later. But it even worked in VR until the web APIs changed a bit. And I just didn't feel like updating it. But I'll fix that later. This whole UI runs in AR, VR just fine. You can position it anywhere in 3D space. It has depth in the UI, all that stuff. And I finally got it. And also, this is my own desktop version of the build. I use it every day. This is the same code base. You just compile it as a native desktop application that runs directly on Metal on Mac, directly on DirectX on Windows, OpenGL on Linux. And yeah, no browser sandbox, no Electron bullshit. This binary is one megabyte in a zip file. We're back to 2,000 in terms of file size. So could this be it? Could this be finally what I wanted, a hyper-fast application stack that works on native and on web? So one of the things that I said is that I didn't have CPU time left, right? So let me show you Chrome trace, a Chrome profile. So it's on performance. I'm going to record. Do this. Stop. So as you can see, I can't even see my CPU code almost. It's like half a millisecond to do a color pick in this UI, as opposed to this is the, let's see. Let's see if I can get the same. No. I mean, the tech stack is completely different, right? I'm profiling functionality here. Because that is what, in the end, matters. It is not about comparing JavaScript versus WebAssembly in an inner loop. It's about comparing the whole stack. So here we go. I'm going to color pick this. And I'm going to see. Yeah, so this is what the, I don't know, it's a bit small, but this is what the JavaScript profile looked like. And these slash things are my worker thread. And now this is an M1 Mac, so it's gotten faster since I abandoned this project. So now it's only half. But back in the day, this was capping at 100% CPU. So yeah, I finally got to the point where I can do useful things with my UI, instead of just doing the UI. So what is it that we built? It's a UI stack. So it's a widget set. It has a React-like render model. It is its own IDE and code editor built on top of that stack. And our final piece to build this into something useful that I can offer to you all is going to be a design tool. Because hand coding UI is something that I would love to delegate to the past. You want to use a proper visual designer. So this whole system is designed to support a visual designer. And that means that we have a DSL. And a DSL is a domain-specific language. HTML is a DSL. CSS is a DSL. We have to have something like that as well, because I don't want to do it all in Rust. If you build a visual design tool, the visual design tool manipulates the data structure that is your UI. And in this case, that data structure is something custom that has properties of HTML, CSS. It has inheritance. It's actually my attempt to finally fix the crap. But we'll see how far that went. Here's, for instance, the color picker in line now. This kind of stuff where you have a code editor that's extendable with all sorts of visual editors is one of the reasons I want a fast render model. Like the code folding just folds everything with it still as well. But why? Originally, my original motivation of getting into programming was having fun with computers. I wanted to make it draw graphics, make sounds, do fun things, and not fight with webpack or I don't know what kind of deferred problem you have as a web developer. And that's why I want to show you our first example application, because it's fun that the IDE is an example. But it's a bit of a domain-specific example. And that's why I built FunAudio. And FunAudio is a post-to-logic audio. It's an audio application that uses our audio API on Mac. However, I have accessed, because in Rust, you can access all native APIs. I accessed the audio unit API. That means that I can load native instrument. I can load native instrument components straight into the UI. This will also build to web, but then you don't get all these nice features, because this is what the web sandbox limits you to. You cannot do this. There's just no way. And with Rust, you get the power to be limited on the web, but get to deploy and the power on desktop. And that is why I get excited by this tech stack. It's a very different approach to putting the web on desktop. It's putting the desktop on web. About four minutes left. So what is the future? When can you guys use this? We're having an open source release to crates, which is NPM for Rust, so that you can very easily install the stack in about three months, including some documentation. And we're ramping up to release the design tool ASAP. And for that, you can open makepad.dev. That was the web thing that you just saw. You can open it right now on your phone. This is the GitHub repo. Everything that you've seen so far is MIT in the GitHub repo. And Twitter for myself and the product. So I have about four minutes left. Let's see if I can add some interesting bits. So yeah, makepad on web, right? So if you deploy these kind of applications to web, size matters, right? You don't want it to be 16 megabytes over the wire. That's why I'm very excited that this thing, as you see it here, is 500 kilobytes over the wire, including all dependencies. So I mean, that's broadly compressed. But still, 500 kilobytes to deploy this whole thing, and it loads in under a second time to interactive. So it's very, very viable to do things this way. It's not like compiling a ginormous amount of code to the web, as a lot of C++ applications are doing. And similarly, if you look to the future of AR and VR, this demo had a collaborative VR. So you could put your quest on, open this URL, and you'd be collaboratively in this IDE space whilst live editing an object in between. And I was very impressed with that as well for web VR, because I was like, ah, it's going to be janky, right? Because VR is so performance sensitive, if your garbage collector hiccups up, or you're going to get sick, because the frame is going to start to judder. But if you're super disciplined about it, and you use WebAssembly only, I could get it completely rock solid on web VR. And this is also when that one millisecond of CPU time and doing stuff really starts to matter, because this is a millisecond of CPU time on an M1 Max, not on a crappy Android chip that is in a VR headset. So that one millisecond that I showed you, you should multiply by 10 to have your full application domain for other devices. And so don't think that I'm over-optimizing that one millisecond. It's actually the space you have to reach your entire audience. So yeah, I think that is the end of my talk. I think we can move to questions. Thank you. I really love the question that is, oh, did it go away now? Ah, yeah. So do you get to work on this full time, or is there a sponsor or company behind? So for the first many years, I did consulting, and I did this as an iterative cycle every time, because I restarted six times. More recently, when I started working with my co-founders, we have an angel-funded startup situation. But it's, yes, full time. Full time. Wow, that's really cool. Any plans to donate it to a foundation at one point? Absolutely. I mean, this is an open source stack, right? The only thing that we're going to commercialize is the design tool that sits on top of it. All of the libraries, all the dependencies are going to be unencumbered MIT, so that everything you build with this, you can pick any license you want. So as this becomes popular and successful, hopefully, then we can definitely use one of those constructs that many open source projects use. Super, very interesting. I hope that answers the question from the person who asked. Is it also possible to import 3D models or textures from Blender, for instance? That's definitely a higher level tool. Yes, absolutely. But someone has to write the importer. It's a custom render stack with a shader model that is unified across all platforms. So it's not GLSL. It's something that looks like GLSL, semantically GLSL, but it cross compiles to all the backends, Metal, HLSL, and WebGL, and in the future, also WSL. So it's a custom engine. It's geared for UI, but it can technically do all the things you could do in 3JS. But we're going to need to develop the tool chains around that. Very cool, thank you. Did you start with building an MVC for the Rust implementation to know if it would really outperform JavaScript? Or was this just sort of fingers crossed, hoping that it would be better? This was fingers crossed. Essentially, this is not something I can benchmark dry very much. I just had to build it. And as we got further, I started getting more excited about half a millisecond of CPU time doing anything. And this means that the amount of bandwidth I have for this UI, I could render. The startup time for this IDE, in terms of spawning up the UI, is a millisecond. Like, I could draw tens of these things at the same time on your phone, and it wouldn't matter. So yeah, it is definitely something I hoped. But I would say it's better than I thought. Awesome. I see your question whether it will only continue to run in the browser. You did show the desktop version that you used yourself. Is that going to be available too? The idea is that we want the power of both. You want to easily deploy to web. And if you're constrained, use the desktop compile. And maybe he means mobile, or they mean mobile. Mobile support will come later, because I think focusing on the web deploy target first is, I think, a good start. Because it's a very big space, especially because we're bridging the APIs to Rust. That's a lot of work. But mobile will come later. Very cool. Do you see a future for Rust-based UIs? Do you think it will be? I think it's absolute garbage. No. That was a very unexpected answer. All right. Thank you very much. I think that's about the time that we have. And we're going to move to lightning talks. Thank you so much for your time. All right. Thank you. Woo! Thank you.
