Type-safe bindings for Node.js with Rust and WebAssembly

Rate this content

This talk will teach you how to write performance-critical Node.js modules without the burden of distributing platform-dependent artifacts and using the C/C++ toolchain. You will discover how to smoothly integrate Rust code into your Node.js + TypeScript application using WebAssembly. You will also learn how to avoid the typical WebAssembly serialization issues, and understand when other alternatives like Neon or Napi.rs are preferable. Together, we will cross the language bridge between Rust and Node.js while preserving the familiar DX you're used to.

22 min
17 Apr, 2023


Sign in or register to post your comment.

AI Generated Video Summary

This Talk explores TypeScript bindings for NodeJS with Rust and WebAssembly, providing an alternative approach for creating native NodeJS modules and automatically generating types. It delves into the use of WebAssembly and Rust for TypeScript modules, showcasing how Rust functions can be defined and imported using the wasm.bindgen library. The Talk also highlights the challenges of string conversion between Rust and JavaScript, the limitations of supporting Rust data types in JavaScript, and the seamless integration of Rust functions into TypeScript apps using tspy. It concludes with the recommendation of TSFI for type-safe bindings and showcases its usage in a TypeScript-based full-text search engine with WebAssembly support.

1. Introduction

Short description:

This is TypeScript Bindings for NodeJS with Rust and WebAssembly. We'll be looking for an alternative, easier approach for creating native NodeJS modules, while also automatically generating types for them.

Hi, everyone, and thanks to Node Congress for having me. This is TypeScript Bindings for NodeJS with Rust and WebAssembly. We'll be looking for an alternative, easier approach for creating native NodeJS modules, while also automatically generating types for them.

A little bit about me. I'm Alberto Schibel. I'm from Venice, Italy. I'm a software engineer at Prisma where I ported several Rust modules to WebAssembly. I'm also a consultant working with NodeJS, TypeScript, and Rust. You can find me online at j.com.io. You can also find the slides for this talk on my GitHub page.

2. WebAssembly and Rust for TypeScript Modules

Short description:

WebAssembly, or WASM, is a low-level abstraction for the CPU your code is running on. It's a fast, compact bytecode designed for near-native speed and optimized for fast start-up time and small memory footprint. It's a portable compilation target for many languages, including Rust. Rust is consistently voted as the best language for WebAssembly. Let's see how we can create TypeScript modules from it.

So, elephant in the room, what is WebAssembly and how's that useful? Well, WebAssembly, or WASM, is basically a low-level abstraction for the CPU your code is running on. It's fast, compact bytecode in the sense that it's a portable binary format for a virtual machine that models loads and stores of numbers inside linear memory. It's designed for near-native speed and is also optimized for fast start-up time and small memory footprint. It was created by browser vendors to port C++ code to the web without performance degradation and now it's a portable compilation target for many languages including Rust, Go and many others.

That means you can compile your code to WASM once and then you can run the same compiled artifact on different platforms. For instance Node.js supports WebAssembly since version 8 and now you can import a WASM module exactly like you would import a standard npm package. And why is a single portable compilation target useful? Consider Node.js, a popular NAPI add-on that compiles SAS styles to CSS. To support multiple system configurations, architectures and even Node.js versions this library needs to be compiled separately for each of these configurations. This means 35 different compilation targets and it makes every new deployment a time and resource consuming task. Those of you who use Prisma will probably know that we're on a similar situation with TypeScript CLI and a library that downloads some compiled Rust binaries on demand. This is what made us consider WebAssembly and adopted as much as we to simplify our deployment process. And if you ever tried to write native Node.js add-ons yourself, you probably know that it's not a straightforward process. Sometimes, no jib fails with cryptic error messages and, frankly, the tooling necessary to build and import C++ modules isn't as human friendly as what Node.js developers are accustomed to. So, this is perhaps one of the major reasons why Rust is consistently voted as the best language for WebAssembly.

So, let's see how we can create the TypeScript modules from it. And for those of you new to Rust, let's define some baseline glossary, right? So, anything you use a package of json for, you would put in a cargo.toml file in a Rust project. What you usually call npm packages are crates in Rust and you operate on them via the cargo CLI. For instance, compiling Rust code, you would use the cargo build command and specify the completion target. In our case, it's Wasm 32 unknown, unknown. It uses a 32-bit addressing space and isn't tied to any particular OS vendor or CPU architecture. If we want to move around more than purely numeric data across the WebAssembly bridge, we're going to need a binding tool. It's both a CLI and Rust library. When you install it, you should specify a particular version, because it doesn't yet follow semantic versioning. This means that the version you specify in your cargo.toml file should match the version of Wasm you have installed on your machine. Moreover, to support WebAssembly, you need to mark your create type as c.lib. It will tell Rust to compile your code as a dynamic library that can be loaded by a C-compatible runtime, like Node.js. So, compiling Rust WebAssembly is a two-step process. First, you need to run cargo.build to create a compiled WebAssembly artifact, which will have the dot wasm extension, and then you will add wasm.bindgen to generate the Node.js and TypeScript bindings you will use to import the compiled wasm module. Of course, if wasm.bindgen supported all the commonly used Rust data structures and TypeScript conventions, this talk will already be over, and clearly that's not the case. So let's see how we can work around this.

3. Rust Functions and WebAssembly

Short description:

Let's define Rust functions that take a number, duplicate it, and return it. Rust syntax can be overwhelming, but we can import the wasm.bindgen library and generate bindings for the functions. We have functions for 64-bit integers and floating point numbers. The generated TypeScript declaration preserves function names and maps number types accordingly.

So, for our first example, let's first see how we can define Rust functions that take a number, duplicate it and return it to the caller. I know that Rust syntax can be overwhelming, so bear with me. We first import the wasm.bindgen library. We then tell Rust to generate bindings for the function that follows, which will be compiled to WebAssembly. And whenever you see a code with a hashtag and a square bracket, that means it's a Rust macro, a special kind of function that expands to generated code at compile time. We define a public function, duplicate underscore U64 that takes an assigned 64-bit integer, multiplies it by two, and returns it. The other function is similar, but it uses floating point numbers instead. And wasm.bindgen generates the following type script declaration for us that you see at the bottom. We see that function names are preserved as is. U64 numbers are mapped to begins. And F32 numbers are plain numbers in TypeScript because it doesn't really have a dedicated floating point type.

4. String Conversion Functions

Short description:

Here's an example with strings. We have a function that converts a string to uppercase and another function that converts a 64-bit signed integer to a string representation. It's important to note the encoding differences between Rust and JavaScript for strings.

Here's a similar example with strings. To the left, we have a two upper case function that takes a string and returns a new string in all caps. Observe that in this case, we specify a custom name for the function for the JS bindings and we use the WasBinds macro for that. To the right, we have an NtlString function that takes a 64-bit signed integer and returns a string representation of it. Notice that strings in Rust are UTF8 encoded. However, in JavaScript, they are UTF16 encoded. And this is something you need to be aware of, especially if you're manipulating strings that may contain emojis or non-Latin characters.

5. Using Functions in TypeScript

Short description:

What happens when we try to use these functions in TypeScript? If we pass compatible types, they work as expected. But if we escape TypeScript validations and pass incompatible types, we'll get runtime errors. Complex data structures like structs have unexpected behavior when generating TypeScript bindings. The generated code leaks internal details and lacks a constructor. Wrapping strings in a struct causes compilation errors. Enums work one-to-one with TypeScript enums, but they can be problematic. Discriminated unions, or target unions, are not supported by WasBindgen.

What happens when we try to use these functions in TypeScript? Well, if we pass types compatible with the TypeScript declarations, they work as expected at runtime. But if we escape from the TypeScript validations by disguising a string as a big int, and we call n to string function with that? Well, in that case we'll get a runtime syntax error because the function expects a number but it's being called with a string.

What if we need more complex data structures? Here's an example with our scholars struct which wraps values like numbers, characters, or booleans. Say we want a function that extracts the value of one of the fields namely the letter. If we were to manually write the typescript bindings for this we will define scalars as a typed dictionary which we call construct in place and we type the letter field as a string because typescript doesn't really distinguish between single character and multi-character strings. However, this is not what WasBindsNGenerates and this is what it gets, what it creates and although the four struct members have the types we expect, we actually get a scalar class definition, not a dictionary type. Moreover, we see some internal details that are leaking out to the generated code and namely that's the free method which doesn't take any argument and doesn't return any value. This is not something we wrote in our Rast type. This is something that WasBindsNGenerates.

Do you also notice that something else is missing? Well, this class doesn't have a constructor so how do we create instances of it from Note.js? Well, we can attempt to call the default JS constructor and we can assign the fields manually. We also need to specify a default implementation for this free method. However, if we do this and we pass this Scalar class instance to the get letter function, well, this will fail at runtime with a cryptic error. NullPointerPassToRust. And it turns out that we can actually fix this by manually defining a constructor in Rust which takes the four struct members as arguments using the constructor macro and calling the constructor from TypeScript. It's clear this is not the best experience we can get, right? As it requires boilerplate code and is not ergonomic for TypeScript devs. By the way, notice that the letter field is automatically truncated to a single character string, although we initialize it with a longer string. And what happens if we wrap strings in a struct, similarly to how we did with the scholars. This code will not compile. That's because strings and rust are noncopyable and WasBindzen will need to copy strings around. And one way to get around the problem is making WasBindzen clone the string with a dedicated macro attribute, getter with clone, but this is not something a TypeScript developer should be concerned with. It's an internal detail we don't need to be aware of. We still don't need to be aware of. Also, we still get a class binding rather than a dictionary type and we've seen how cumbersome and awkward that is to use. How about enums, while C-style enums are translated one to one to TypeScript enums, so we can see that WasBindzen works out of the box in this case. However, enums are often considered a bad practice in the TypeScript community as they are a little bit hard to reason around because the JavaScript runtime doesn't have any notion of enums, right? So, that could lead to unexpected bugs. Ideally, we would prefer to get a union of literal types instead, like the one we see at the top right. How about discriminated unions, or target unions? They are a popular pattern in TypeScript, especially when encoding algebraic data types. And it turns out that Rust supports them in the form of enum variants. In this example, we have either type that at a given time encodes a successful numeric result, with an okay constructor, or a failure message, with an error constructor. However, enum variants are not supported by WasBindgen as the compiler error message tells us, so we cannot really use them as they are.

6. Supporting Rust Data Types in JavaScript

Short description:

WasBindgen provides partial support for vectors, but only for numeric types. It's limited and not ideal for TypeScript developers. Serde is a non-standard library that provides serialization and deserialization utilities for Rust. It allows us to work with more Rust data types in JavaScript by exposing functions that consume and return JSON-encoded strings. The SerDeWasmBindingCrate offers a more efficient approach with native binary integration, supporting enum variants, generating vectors, and maps. However, using js value arguments in Rust functions sacrifices type safety.

Finally, WasBindgen provides partial support for homogeneous vectors, but only for numeric types, which are translated to typed array instances. And they are essentially only useful when manipulating raw binary data. They are quite far from the standard general-purpose arrays we usually want. Also, vectors of non-primitive types, nested vectors, or tuples are not supported at all.

So, WasBindgen provides the basic tools to port Rust libraries to Node.js. But it's neither ergonomic nor ideal for typescript devs and is overall quite limited. Can we do better than this?

Well, the first non-standard library that every Rust developer usually encounters is Serde, which provides macros and utilities to serialize and deserialize common Rust-types to and from several formats with minimal boilerplate. One first step to support more Rust data types in JavaScript is exposing functions that consume and return JSON-encoded strings, which we can then parse and stringify in Rust via Serde's serialized and deserialized traits. I've listed also the dependencies that we need to add to our cargo.toml file and our versions for everyone's convenience.

Here's how it would work. We first import Serde we apply them to a Rust struct or enum to make it automatically serializable and deserializable. Think of those traits as interfaces needed to translate data structures to formats like JSON and think of the derive macro as something that implements those traits for us. Then we define a public string to string function with the WasmBindgen macro we already used to. Next we parse the input string which we assume being JSON encoded into the scholars struct we defined above. We compute the result and we serialize it back to JSON. And then we return this to the caller. Notice that the TypeScript binding is technically typed but it's not very useful as we could pass any JSON or even a string that is not a JSON at all and TypeScript will still accept it at compile time although it will result in an error at runtime. JSON serialization can be expensive in practice so the SerDeWasmBindingCrate came up with a more efficient approach providing a native binary integration of SerDe with WasmBinding. The project is currently maintained by CloudFlare by the way and again since it relies on SerDe we get support for plenty of other types which we can use in JavaScript. Notable differences from the plain WasmBinding approach is that enum variants can be translated to tagged unions. We get generating vector support as well as support for maps.

Similarly to the previous example, we can define a scalar struct. Actually this is a subset of the example. We can expose a public function that takes a scalar's value as an input and we return its letter field to WasmBinding. However, notice that this time the Rust arguments are typed as js value which models any value that can be passed to or that can be received from JavaScript. Then it's up to us to cast these js values to actual types. And we will do that by using the Std WasmBinds from value utility and then we can cast this result back to js value. And without digging into too much too many details, well, we see that the function signature tells us that the result is either a js value or a specific error type provided by Std WasmBinds. Namely, that's a WasmError. However, if we use this approach, we lose type safety entirely, as the js value can literally be any value.

7. Seamless Integration with tspy

Short description:

We discovered tspy, a magical tool that generates type-safe and ergonomic bindings for seamlessly integrating Rust functions into TypeScript apps with WebAssembly. It eliminates the need for manual casting and provides strong late type bindings. We'll demonstrate its usage with Enum variants and the serialization process. WebAssembly is ideal for CPU-intensive tasks and complex logic, but lacks input-output support. TSFI is the best solution for type-safe bindings, although it heavily relies on macro magic. Generic containers like vectors and hash maps require specifying the generic type and wrapping them in a struct or enum variant. Check out the example of TSFI usage in Lira, a TypeScript-based full-text search engine with WebAssembly support.

So it's typed as any in TypeScript and it's not really that useful. So we started this journey in an effort to seamlessly integrate Rust functions for data structures into TypeScript apps with WebAssembly. And it looks like we should give up. Unless maybe there is a magical tool that could help us by generating type safety and ergonomic bindings.

Well, thankfully that tool exists. It's called tspy and I honestly love it. It supports everything we've seen so far but doesn't need any manual casting and it comes with strong late type bindings. We'll see in a second that we're going to need a little bit more macros to make things work. But still a huge improvement over the previous approaches. Notice that also we need to install tspy with the JS feature flag, which will give us a native JavaScript integration. Otherwise it will use JSON serialization by default.

As a demonstration, we will readapt the previous example using Enum variants, which we wanted to be translated to target unions in TypeScript. So, we see that we need to derive SerDes traits, as well as the new tspy trait. We also need to use a new tspy macro to tell Wasp bind to compile some data types, you know, a data type that is otherwise unsupported by Wasp binding. In fact, Wasp ABI stands for WebAssembly Application Binary Interface and it describes how to call functions between languages in WebAssembly. We then define the familiar either variant with a twist that is common to all the approaches that use SerDe. We need to tell Rust how to serialize this in a variant because, you know, this could happen in a plethora of ways and to get idiomatic tag unions like the ones we see on the right, we have to tell Rust that the variant name should be associated with its dominant key, namely underscore tag, and that the content of the variant, which is defined between the constructor parenthesis, should be associated to a property named value. We can then define a function that for instance takes an instance on either and returns its string representation. Notice that it doesn't really require us to write any type casting boilerplate code and it translates to a clean TypeScript definition. Just like SerD vs BindGen, we can define an either value in JavaScript without needing any constructor. We can just create it on spot as a dictionary. But this time we get TypeScript guarantees, so we can leverage TypeScript's compiler to avoid writing typos in our data types. So let's wrap up what we've learned so far.

WebAssembly is here to remain, and it's good for CPU-intensive tasks that would otherwise be too slow in pure JavaScript, or for parting already existing complex logic to the web. Think about Figma. However, it currently provides almost no input-output support. So if you need to interact with the outside world from your functions, you'd better stick with the NAPI for the moment. We've iterated through several approaches to port Rast functions to Node.js and observed there are limitations or awkward developer experience, especially for TypeScript ads. We've finally seen that the best solution for type-safe bindings, TSFI is still relatively new. One caveat is that its source code heavily relies on macro magic, right? And that could be a deal-breaker for someone. Also for any set of the approach, and that includes TSFI, you can't just use generic containers like vectors or hash maps directly in a function that you bind to WebAssembly. You actually first need to specify the generic type. So, you have to do, you have to say, a vector of strings and then you have to wrap it into a struct or enum variant that you then expose to SerDe via the serialized or de-serialized traits and then you use that in your function. And if you want to see some example of TSFI being used in the wild, you can check a PR online that introduced WebAssembly support to Lira, a full-text search engine written in TypeScript by Mikhail Eriva, which I believe spoke here at Node Congress as well. Michael, oh, well, sorry, Mikhail Eriva was quite happy with the performance improvements. And that's it for me. I'm Bertos Ghebel, you can find me on Twitter and GitHub, you can also find additional material and code samples for this talk on my repository, node-congress-2023.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Vue.js London Live 2021Vue.js London Live 2021
8 min
Utilising Rust from Vue with WebAssembly
Rust is a new language for writing high-performance code, that can be compiled to WebAssembly, and run within the browser. In this talk you will be taken through how you can integrate Rust, within a Vue application, in a way that's painless and easy. With examples on how to interact with Rust from JavaScript, and some of the gotchas to be aware of.
JSNation Live 2021JSNation Live 2021
29 min
Making JavaScript on WebAssembly Fast
JavaScript in the browser runs many times faster than it did two decades ago. And that happened because the browser vendors spent that time working on intensive performance optimizations in their JavaScript engines.
Because of this optimization work, JavaScript is now running in many places besides the browser. But there are still some environments where the JS engines can’t apply those optimizations in the right way to make things fast.
We’re working to solve this, beginning a whole new wave of JavaScript optimization work. We’re improving JavaScript performance for entirely different environments, where different rules apply. And this is possible because of WebAssembly. In this talk, I'll explain how this all works and what's coming next.
Node Congress 2022Node Congress 2022
26 min
It's a Jungle Out There: What's Really Going on Inside Your Node_Modules Folder
Do you know what’s really going on in your node_modules folder? Software supply chain attacks have exploded over the past 12 months and they’re only accelerating in 2022 and beyond. We’ll dive into examples of recent supply chain attacks and what concrete steps you can take to protect your team from this emerging threat.
You can check the slides for Feross' talk

Vue.js London 2023Vue.js London 2023
30 min
Stop Writing Your Routes
The more you keep working on an application, the more complicated its routing becomes, and the easier it is to make a mistake. ""Was the route named users or was it user?"", ""Did it have an id param or was it userId?"". If only TypeScript could tell you what are the possible names and params. If only you didn't have to write a single route anymore and let a plugin do it for you. In this talk we will go through what it took to bring automatically typed routes for Vue Router.

Workshops on related topic

React Advanced Conference 2021React Advanced Conference 2021
174 min
React, TypeScript, and TDD
Featured WorkshopFree
ReactJS is wildly popular and thus wildly supported. TypeScript is increasingly popular, and thus increasingly supported.
The two together? Not as much. Given that they both change quickly, it's hard to find accurate learning materials.
React+TypeScript, with JetBrains IDEs? That three-part combination is the topic of this series. We'll show a little about a lot. Meaning, the key steps to getting productive, in the IDE, for React projects using TypeScript. Along the way we'll show test-driven development and emphasize tips-and-tricks in the IDE.

React Advanced Conference 2022React Advanced Conference 2022
148 min
Best Practices and Advanced TypeScript Tips for React Developers
Featured Workshop
Are you a React developer trying to get the most benefits from TypeScript? Then this is the workshop for you.
In this interactive workshop, we will start at the basics and examine the pros and cons of different ways you can declare React components using TypeScript. After that we will move to more advanced concepts where we will go beyond the strict setting of TypeScript. You will learn when to use types like any, unknown and never. We will explore the use of type predicates, guards and exhaustive checking. You will learn about the built-in mapped types as well as how to create your own new type map utilities. And we will start programming in the TypeScript type system using conditional types and type inferring.
Node Congress 2023Node Congress 2023
109 min
Node.js Masterclass
Have you ever struggled with designing and structuring your Node.js applications? Building applications that are well organised, testable and extendable is not always easy. It can often turn out to be a lot more complicated than you expect it to be. In this live event Matteo will show you how he builds Node.js applications from scratch. You’ll learn how he approaches application design, and the philosophies that he applies to create modular, maintainable and effective applications.
: intermediate
Node Congress 2023Node Congress 2023
63 min
0 to Auth in an Hour Using NodeJS SDK
Passwordless authentication may seem complex, but it is simple to add it to any app using the right tool.
We will enhance a full-stack JS application (Node.JS backend + React frontend) to authenticate users with OAuth (social login) and One Time Passwords (email), including:
- User authentication - Managing user interactions, returning session / refresh JWTs
- Session management and validation - Storing the session for subsequent client requests, validating / refreshing sessions
At the end of the workshop, we will also touch on another approach to code authentication using frontend Descope Flows (drag-and-drop workflows), while keeping only session validation in the backend. With this, we will also show how easy it is to enable biometrics and other passwordless authentication methods.
Table of contents
- A quick intro to core authentication concepts
- Coding
- Why passwordless matters
- IDE for your choice
- Node 18 or higher
JSNation Live 2021JSNation Live 2021
156 min
Building a Hyper Fast Web Server with Deno
Deno 1.9 introduced a new web server API that takes advantage of Hyper, a fast and correct HTTP implementation for Rust. Using this API instead of the std/http implementation increases performance and provides support for HTTP2. In this workshop, learn how to create a web server utilizing Hyper under the hood and boost the performance for your web apps.

TypeScript Congress 2022TypeScript Congress 2022
116 min
Advanced TypeScript types for fun and reliability
If you're looking to get the most out of TypeScript, this workshop is for you! In this interactive workshop, we will explore the use of advanced types to improve the safety and predictability of your TypeScript code. You will learn when to use types like unknown or never. We will explore the use of type predicates, guards and exhaustive checking to make your TypeScript code more reliable both at compile and run-time. You will learn about the built-in mapped types as well as how to create your own new type map utilities. And we will start programming in the TypeScript type system using conditional types and type inferring.
Are you familiar with the basics of TypeScript and want to dive deeper? Then please join me with your laptop in this advanced and interactive workshop to learn all these topics and more.
You can find the slides, with links, here:
And the repository we will be using is here: