Hi, I'm Lin Clark and I make code cartoons. I also work at Fastly. Which is doing a ton of cool things with web assembly to make better Edge Compute possible. And I'm a co-founder of the Bytecode Alliance. We're working on tools for WebAssembly ecosystem that extends beyond the browser. And that is one of those tools that I wanted to talk to you about today.
But first, I want to give you a heads up. This talk is structured a bit differently than speaking experts would tell me I should be structuring this presentation. I'm going to start with telling you how we're making this work at all. And once you've heard that you might not be on board, you might think that this is a pretty ridiculous idea. So that's why I'm going to explain why. Mostly, why you would actually want to do this. And then once you're bought in, and I know you'll be bought in, then I'm going to come back and explain exactly how it is that we're making this fast.
[09:00] So one part of this is application initialization. For any project, the contractor needs to take a look at the work that the client wants it to do, and then set up the resources that it needs in order to complete that job. So for example, the contractor reads through the project briefing and other supporting documents and turns them into something that it can work with. So this might be something like setting up the project management system with all of the documents stored and organized and breaking things into tasks that go into the task management system.
In the case of the JS engine, this work looks more like reading through the top-level of the source code and parsing functions into Bytecode, or allocating memory for the variables that are declared, and setting values where they're already defined.
[09:46] So that's application initialization, but in some cases there's also engine initialization and you see this in contexts like Serverless. The JS engine itself needs to be started up in the first place, and built-in functions need to be added to the environment.
I think of this like setting up the office itself, doing things like assembling the IKEA chairs and tables and everything else in the environment, before starting the work. Now, this can take considerable time and that's part of what can make the cold start such an issue for serverless use cases.
[10:23] Once the initialization phase is done, the JS engine can start its work. This work of running the code. And the speed of this part of the work is called throughput, and this throughput is affected lots of different variables. So for example: which language features are being used, whether the code behaves predictably from the JS engine's point of view, what sorts of data structures are used, and whether or not the code runs long enough to benefit from the JS engine's optimizing compiler.
So these are the two phases where the JS engine spends its time. Initialization and Runtime. Now, how can we make the work in these two phases go faster?
[11:10] Let's start with initialization. Can we make that fast? And spoiler alert. Yes, we can. We used a tool called Wizer for this and I'll explain how that works in a minute. But first I want to show you some of the results that we saw. We tested with a small markdown application. And using Wizer, we were able to make startup time six times faster. If we look in more depth at this case, about 80% of this was spent on engine initialization. And the remaining 20% was spent on application initialization. And part of that is because this markdown render is a very small and simple application. As apps get larger and more complex, application initialization time just takes longer. So we would see even larger comparative speed ups for real-world applications.
Because this linear memory is so self-contained, once all of the values have been filled in, we can just take that memory and attach it as a data section to a Wasm module. When the JS engine module is instantiated, it has access to all of the data in the data section. Whenever the engine needs a bit of that memory, it can copy the section, or rather the memory page, that it needs into its own linear memory. With this, the JS engine doesn't have to do any setup when it starts up. All of this is pre-initialized, ready and waiting for it to start its work.
[13:11] Currently, we attach the data section to the same module as the JS engine. But in the future, once WebAssembly module linking is in place, we'll be able to ship the data section as a separate module. So this provides a really clean separation and allows the JS engine module to be reused across a bunch of different JS applications.
[14:06] It's kind of like the JS engine contractor doesn't need to set up its own office at all. It just gets this travel case shipped to it. And that travel case has the whole office, with everything in it all set up and ready to go for the JS engine to just get to work. And the coolest thing about this is that it doesn't rely on anything that's JS dependent. It's just using an existing property of WebAssembly itself. So you could use the same technique with languages like Python, Ru, Lua and other run times, too.
[15:24] Now, as I said before, it's not possible to JIT compile code within pure WebAssembly module at the moment. But it turns out that we can apply some of the same thinking that comes with Just-in-Time compilation to an ahead-of-time compilation model.
So one optimizing technique that JITs use is inline caching, which I also explained in my first series about WebAssembly. When the same bit of code gets interpreted over and over and over again, the engine decides to store its translation for that bit of code to reuse next time. And the stored translation is called the stub. Now these stubs are chained together into a linked list. And they're based on what types are used for that particular invocation. The next time that the code is run, the engine will check through this list to see whether or not it actually has a translation that is available for those types. And if so, it'll just reuse the stub.
[16:22] Because IC stubs are commonly used in JITs, people think of them as being very dynamic and specific teach program. But it turns out that they can be applied in an AOT context, too.
Now this is just one example of a potential optimization that we can make. Right now, we're in the same kind of position that the browser JS engines were in in the early days. When they were first experimenting with Just-in-Time compilers in the first place. We still have a lot of work to do to find the clever shortcuts that we can use in this context. But we're excited to be starting that work and excited for the changes to come.
If you're excited like we are about this and want to contribute to the optimization efforts, or if you want to try to make this work for another language like Python or Ru or Lua, we'd be happy to hear from you. You can find us on the messaging platform, Zulip. Feel free to post there if you want to ask for more info. You can also find links to the projects that I mentioned in my recently published blog post on the Bytecode Alliance blog. I want to say thank you to the organizers for inviting me to speak here today. And thank you all for listening.
[19:17] Mettin Parzinski: Hey, Lin.
Lin Clark: Hi there.
Mettin Parzinski: Good to see you.
Lin Clark: Good to see you too.
[19:22] Mettin Parzinski: So with 55%, we have, "No, but I want to," and 40% said, "No." Does this surprise you? Is this what you were expecting?
[20:31] Mettin Parzinski: You're talking more about that people are using it as a consumer, but not as a developer. Yeah, it doesn't surprise me at all. Even though it's been around for a long time, I actually hop around from client to client every year, and I never hear anywhere where it's used yet here in the Netherlands. So of course, there will be companies, but that's my experience, so it doesn't surprise me.
So let's jump to the Q&A. We have some questions from our audience, and if you still have any questions for Lin, then you can jump to the Community Track Q&A channel in Discord. I want to make one little note. Of course, you made your cartoons and I think everyone here must've fallen in love with your slides, and you will see a big spike in your traffic on the Code Cartoons website. Really great styling of your slides, so I wanted to give some compliments on that.
[21:31] Lin Clark: Thank you. The Code cartoons website has not been updated in a while. I need to actually take care of that. So a great place to find them is the Bytecode Alliance blog or Mozilla Hacks website.
Mettin Parzinski: And on your Twitter, I think?
Lin Clark: Yes. And on my Twitter.
[21:51] Mettin Parzinski: For the latest and greatest from Lin, check her Twitter. The first question is from Alexis. "Do you think that WebAssembly will become a major part of web development, bringing other languages to the web, or will it stay in the area of computational heavy applications?"
[23:47] Lin Clark: I'd be interested. You said managed modules?
Mettin Parzinski: No, they're all highly used modules, standard modules. So I'm thinking this means it means something like Underscore.
[24:35] Mettin Parzinski: Yeah, that would be awesome. Next question is from our attendee, Keith "What would be a good way to incorporate web assembly in an existing application to minimize risk, but start to get comfortable using it and taking advantage of it and how to use WebAssembly in an existing project?"
[24:55] Lin Clark: So it depends on whether or not you're talking about the web or outside of the web. So if you're talking about on the web, you probably want to be using an application where, if you have something that's computationally heavy, then it makes sense to take that little part that is computationally heavy, and port that little bit to WebAssembly. If you're working outside of the web, there are platforms like the one that we have at Fastly, where you can just easily put up a WebAssembly module that is the artifact that you put up to run on a computed edge platform. So in that case, you're just going to start up a service doing WebAssembly. And so, if you have a microservices architecture, then you can do one of your services in WebAssembly.
[26:01] Mettin Parzinski: Awesome. Next question is from Warlock, "What are build sizes in general? Bandwidth cost of using Wasm?"
[26:58] Mettin Parzinski: Okay. Last question we have time for. Next question is from Bartos, "Is snapshotting possible with V8 too or has it only been explored in the context of Wasm?"
[27:22] Lin Clark: So I think one thing that people sometimes don't realize is the snapshotting that you have in V8, that is applications, and that part's not new. Application snapshotting is not new. The snapshot in the engine is something that you wouldn't have without the WebAssembly runtime to do that snapshotting. Because if you're opening up a V8 isolate, what you're starting up is the isolate itself. So, that engine initialization is also part of this.
[28:00] Mettin Parzinski: Awesome. So that was the last question. But I have one question that I feel is too important to skip and it's from Richard S and he says, "Where can I learn more about WebAssembly?"
[28:13] Lin Clark: So, I've done a number of blog posts about WebAssembly. You can find those on the Mozilla Hacks website or on the Bytecode Alliance site. If you want to get involved in WebAssembly, there's a GitHub repo, well GitHub organization where all the standardization happens. That's if you really want to get low level deep into the details.
[28:36] Mettin Parzinski: Nice. All right. Thanks. Lovely. So there's some more questions in the Discord channel that we don't have time for, but I will invite everyone that still has questions for Lin to join Lin on our room on Spatial Chat, where she will be going now. So Lin, thanks a lot for joining me here and enjoy your Spacial Chat speaker room.
Lin Clark: Thank you.