How to build distributed systems in TypeScript

Rate this content

Learn how running your TS in Temporal's runtime gets you extreme reliability

10 min
29 Apr, 2022


Sign in or register to post your comment.

AI Generated Video Summary

This Talk discusses building distributed systems in TypeScript, focusing on an e-commerce store and implementing a create order endpoint. It covers handling retry and error cases, saving order state, handling crashes and duplicate orders, and the process order function and architecture. The Talk also introduces Temporal, a durable code execution framework that simplifies fault-tolerant software development.

1. Building Distributed Systems in TypeScript

Short description:

This is how to build distributed systems in TypeScript. We will work on an e-commerce store and implement a create order endpoint. We will go through all the failure modes and try to make it more reliable. We reserve the inventory, charge the payment, and send the package. If any step fails, we handle it accordingly and consider retrying.

Hi, folks. This is how to build distributed systems in TypeScript. My name is Lauren, and I wrote a book on GraphQL called the GraphQL Guide, and currently I am working as a language runtime engineer at Temporal working on our TypeScript SDK.

So when working in a monolith, we might not have very many distributed systems concerns to be worried about. If we just have a single app server layer and a database that does transactions, then we might just be getting a request from a user, doing a single operation, and then if that fails, then we say to our user, try again. If we have, if we're talking to external APIs and they're an essential part of the business application, like I talk to Stripe and say charge this, and then I update my database, then I have some more problems, like I might not be able to reach Stripe. So I need to retry that. When I retry, I need to do so idempotently, so that I'm not double charging the user. And there are cases in which my database might be out of sync with Stripe's understanding of the world. So Stripe returns success and I can't reach the database and that operation fails or my process dies before I get to that step, then my data is inconsistent.

In this talk, we will be working on an e-commerce store and implementing a create order endpoint. And we will go through all the failure modes and try to make it more reliable, and we won't get to full reliability, but we'll see how much simpler we can make it if we write it in temporal. To get started, we have a serverless function hosted on Vercel, and we get out of the body the item ID, the quantity, and the address that you go to, and we get the user ID out of the jar. And then we take three steps whenever we have a new order. We reserve the inventory with the inventory service. Then we talk to the payment service to charge, and then we talk to the fulfilment service to send the package. And we respond success.

So in a successful case, this logic works fine, but there are a number of things that can go wrong. If we fail to reserve, then we don't want to continue charging and sending packages. If we successfully reserve and then fail to charge, then we not only don't want to send the package, but we also want to unreserve from the inventory so that someone else can buy those items. And then lastly, if reserving and charging is successful, and we get that same package, then we want to refund the charge and unreserve. So let's see what that logic looks like. Get down to here, reserve. If the reservation failed, then we say status 400. Now, when we charge and it fails, we want to talk to the inventory service again and say unreserve this amount of this item. And then finally, send package if that fails, then we want to first refund, then unreserve. Right now, when we have a failure, we send status 400 to the client. But ideally, we would be retrying each of these steps in case the failure was transient, like a network error or the service was temporarily down. But if we retry it the second time, maybe it'll go through. So let's add retrying to each of these steps.

2. Handling Retry and Error Cases

Short description:

We have a retry function that handles service calls and retries them with exponential back off. We need an idempotency token to prevent duplicate calls. We also retry failure calls and catch errors in the response object.

We have a retry function that takes parameters, max attempts, a timeout in intervals. And so for each of these attempts, it tries calling the service, it races between that and the timeout. So if 30 seconds passes and we haven't heard back from the service, then we retry. And each time we retry, we are doing exponential back off. And here we're wrapping each of these service calls in the retry function.

One issue we have now as we are retrying is that we might have multiple successful calls and we need some kind of idempotency token so that the service knows not to do the second time. And the best way to do this is to get it from the client. So we'll add that. Get a request ID out of the body and pass it to each of the services.

We also want to retry these failure calls, like if the refund throws, then we'll neither refund nor will unreserve. So the customer will wind up still with the charge and the inventory is taken. So let's add those. So we're wrapping unreserve with the retry and also refund. Also, when we're speaking of throwing, we're not actually catching throwing, we're just seeing whether the response object had a failure property. So let's also be catching these errors.

3. Saving Order State and Handling Errors

Short description:

We need to save the state of the order processing to a durable store and disk. We use Mongo Client to connect to the DB and update the order record at each step. We also need to handle database errors and compensation logic errors. Additionally, a worker pool is required.

So we get a reservation object if we successfully talked to the service. But if we weren't able to reach the service, then that'll throw and we catch that, save it here and then send it to the user. But what happens if this process crashes? So at this point with all these retries, we might be retrying for hours and we no longer fit in the time window of a Vercel serverless function.

And so as we have these long running processes, at some point, some of them will fail, crash because the server loses power or whatever. And we want to be able to pick back up at the same state we were in so that we can continue processing the order. And right now, the only awareness of what step we're on is in-memory of this code.

So at each step, we want to save to some durable store and a disk somewhere what state we're on, what we've done and what we have left to do. So let's do that. We've got Mongo Client and the different states that an order can be in. We connect to the DB, we insert one in the created state, we get the ID and then we use that to update the same order record after every state. Failed to reserve or reserved, failed to charge, et cetera. There are a number of other problems that we don't have time to fix. We want to make sure that the database client is retrying and we want to handle database errors and we want to handle these compensation logic errors. And we need to have a worker pool.

4. Handling Crashes and Duplicate Orders

Short description:

We need a process to handle crashes and continue processing non-terminal records in the database. Temporal is a system that automatically persists and retries our code, eliminating the need for manual handling. In the Temporal system, we start a process order function using the Temporal SDK. If the request is already started, we inform the user that their order has already been submitted.

So right now, we're inside of a HTTP handler, but if this crashes, we need some other process somewhere to notice that a record in the database is in a non-terminal state. The last time it was updated was long enough ago that I know no one's still working on it, and the worker needs to pick that up and continue. We also probably want to return to the user earlier than the end of this function because all of this retry stuff might be taking minutes or hours.

I don't have time to code this. You probably don't have time to code this, and we shouldn't have to. We should be working at a level of abstraction where everything we do is automatically persisted and retried, and we don't have to worry about it. Unfortunately, that system exists, and it's called Temporal. Let's see what our app looks like in the Temporal system. Here's the new serverless function. We still get the same data out. We do only one step though, and that's client.start. This is using the Temporal SDK up here. We're starting a process order function, and we're providing it the order arguments and the request ID as sort of an ID of the function execution and also an id of potency token, and then respond that we've submitted this. If this has happened, we know that it's durably started and saved, and if we get this already started error, then we know the request was a duplicate. So we say, hey, the user's order has already been submitted.

5. Process Order Function and Architecture

Short description:

We have a simple process order function with steps like reserve, charge, unreserve, and send package. These steps are implemented as activities, which handle potential failures. Each activity function is retried and timed out based on the retry policy. The process order function is persisted, allowing for continuation in the same state if the process dies. The system also supports additional features like timers and sleep. The architecture involves a temporal server, language runtimes, and SDKs.

Now let's look at this process order function. We have each step, reserve, charge. If charge fails, then we unreserve, and then we send package, and if that fails, we refund and then unreserve. So that's very simple. That's all of our logic and it works.

You might be thinking, okay, but these reserve, charge functions, et cetera, must be really complicated. Let's look at them. So these are called activities, and we use activities for anything that might fail. So all this logic doesn't have a risk of failure, but everything it calls does. This is the service reserve, unreserve. You can see it's just the basic service calls. Each of these activity functions is retried and timed out according to my retry policy and whatever options I put here, there's lots of different options. That's just all taken care of by the system.

Further, each step of this process order function is persisted. So if the process dies after this, then a new worker will continue execution in the exact same spot, in the exact same state, and any local variables. We only have order up here, but we could have lots of other ones that we change during the function. They'll all have the same values. You can also do other cool things like sleep for 30 minutes. We're doing the Amazon one click buy. We say reserve and then we set a 30 minute timer, and then we charge. And if we get canceled during those 30 minutes, then we don't charge our send package. We can also sleep for 30 days in a loop if we're trying to monthly charge someone on a subscription.

And the way this works is we have a temporal server, which is open source, and you can host it anywhere. And that does persistence. And we have language runtimes, or SDKs. And the SDK is put up into a client and a worker. And then you have a client that can be used anywhere. Right now we're using it on a serverless function. And the client talks to the temporal server, and the worker talks to the server. And the client says, start this workflow, which is this process order function.

6. Temporal: A Durable Code Execution Framework

Short description:

Temporal is a durable code execution framework that allows you to write code oblivious to faults. It's a huge step forward in software development, working at a higher level of abstraction. Learn more at or contact me via email at or on Twitter at laurenDSR.

And then the workers are constantly pulling and picking up tasks from the server. Like, okay, start this function, start this activity, et cetera. And the worker is reporting back to the server after it completes each step so the server can persist it.

So when people ask me, what's Temporal? I say it's a durable code execution framework. So we run your code in our workers and we do so durably, such that if anything goes wrong, we pick it back up in the exact same state. And so you can write your code in a way that's oblivious to faults. So that's Temporal.

I think it's a huge step forward in software development, working at a higher level of abstraction in which you don't have to worry about liability. If you'd like to learn more, check out That's the TypeScript SDK docs. Feel free to email me, Hit me up on Twitter, at laurenDSR.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Vue.js London 2023Vue.js London 2023
30 min
Stop Writing Your Routes
The more you keep working on an application, the more complicated its routing becomes, and the easier it is to make a mistake. ""Was the route named users or was it user?"", ""Did it have an id param or was it userId?"". If only TypeScript could tell you what are the possible names and params. If only you didn't have to write a single route anymore and let a plugin do it for you. In this talk we will go through what it took to bring automatically typed routes for Vue Router.
TypeScript Congress 2023TypeScript Congress 2023
31 min
Making Magic: Building a TypeScript-First Framework
I'll dive into the internals of Nuxt to describe how we've built a TypeScript-first framework that is deeply integrated with the user's IDE and type checking setup to offer end-to-end full-stack type safety, hints for layouts, middleware and more, typed runtime configuration options and even typed routing. Plus, I'll highlight what I'm most excited about doing in the days to come and how TypeScript makes that possible not just for us but for any library author.
React Advanced Conference 2021React Advanced Conference 2021
6 min
Full-stack & typesafe React (+Native) apps with
Why are we devs so obsessed with decoupling things that are coupled nature? tRPC is a library that replaces the need for GraphQL or REST for internal APIs. When using it, you simply write backend functions whose input and output shapes are instantly inferred in your frontend without any code generation; making writing API schemas a thing of the past. It's lightweight, not tied to React, HTTP-cacheable, and can be incrementally adopted. In this talk, I'll give a glimpse of the DX you can get from tRPC and how (and why) to get started.
TypeScript Congress 2022TypeScript Congress 2022
10 min
How to properly handle URL slug changes in Next.js
If you're using a headless CMS for storing content, you also work with URL slugs, the last parts of any URL. The problem is, content editors are able to freely change the slugs which can cause 404 errors, lost page ranks, broken links, and in the end confused visitors on your site. In this talk, I will present a solution for keeping a history of URL slugs in the CMS and explain how to implement a proper redirect mechanism (using TypeScript!) for dynamically generated pages on a Next.js website.
Add to the talk notes: 
React Summit 2023React Summit 2023
19 min
7 TypeScript Patterns You Should Be Using
In this talk, we will be going over a number of common useful and best-practice-proven TypeScript patterns to use in React 18. This includes how to correctly type component properties, children and return types, using React's built-in types, typing contexts, and the usual enum rant (but constructively).

Workshops on related topic

React Advanced Conference 2021React Advanced Conference 2021
174 min
React, TypeScript, and TDD
Featured WorkshopFree
ReactJS is wildly popular and thus wildly supported. TypeScript is increasingly popular, and thus increasingly supported.
The two together? Not as much. Given that they both change quickly, it's hard to find accurate learning materials.
React+TypeScript, with JetBrains IDEs? That three-part combination is the topic of this series. We'll show a little about a lot. Meaning, the key steps to getting productive, in the IDE, for React projects using TypeScript. Along the way we'll show test-driven development and emphasize tips-and-tricks in the IDE.

React Advanced Conference 2022React Advanced Conference 2022
148 min
Best Practices and Advanced TypeScript Tips for React Developers
Featured Workshop
Are you a React developer trying to get the most benefits from TypeScript? Then this is the workshop for you.
In this interactive workshop, we will start at the basics and examine the pros and cons of different ways you can declare React components using TypeScript. After that we will move to more advanced concepts where we will go beyond the strict setting of TypeScript. You will learn when to use types like any, unknown and never. We will explore the use of type predicates, guards and exhaustive checking. You will learn about the built-in mapped types as well as how to create your own new type map utilities. And we will start programming in the TypeScript type system using conditional types and type inferring.
TypeScript Congress 2022TypeScript Congress 2022
116 min
Advanced TypeScript types for fun and reliability
If you're looking to get the most out of TypeScript, this workshop is for you! In this interactive workshop, we will explore the use of advanced types to improve the safety and predictability of your TypeScript code. You will learn when to use types like unknown or never. We will explore the use of type predicates, guards and exhaustive checking to make your TypeScript code more reliable both at compile and run-time. You will learn about the built-in mapped types as well as how to create your own new type map utilities. And we will start programming in the TypeScript type system using conditional types and type inferring.
Are you familiar with the basics of TypeScript and want to dive deeper? Then please join me with your laptop in this advanced and interactive workshop to learn all these topics and more.
You can find the slides, with links, here:
And the repository we will be using is here:
TypeScript Congress 2023TypeScript Congress 2023
131 min
Practice TypeScript Techniques Building React Server Components App
In this hands-on workshop, Maurice will personally guide you through a series of exercises designed to empower you with a deep understanding of React Server Components and the power of TypeScript. Discover how to optimize your applications, improve performance, and unlock new possibilities.
During the workshop, you will:
- Maximize code maintainability and scalability with advanced TypeScript practices
- Unleash the performance benefits of React Server Components, surpassing traditional approaches
- Turbocharge your TypeScript with the power of Mapped Types
- Make your TypeScript types more secure with Opaque Types
- Explore the power of Template Literal Types when using Mapped Types
Maurice will virtually be by your side, offering comprehensive guidance and answering your questions as you navigate each exercise. By the end of the workshop, you'll have mastered React Server Components, armed with a newfound arsenal of TypeScript knowledge to supercharge your React applications.
Don't miss this opportunity to elevate your React expertise to new heights. Join our workshop and unlock the potential of React Server Components with TypeScript. Your apps will thank you.
Node Congress 2021Node Congress 2021
245 min
Building Serverless Applications on AWS with TypeScript
This workshop teaches you the basics of serverless application development with TypeScript. We'll start with a simple Lambda function, set up the project and the infrastructure-as-a-code (AWS CDK), and learn how to organize, test, and debug a more complex serverless application.
Table of contents:
        - How to set up a serverless project with TypeScript and CDK
        - How to write a testable Lambda function with hexagonal architecture
        - How to connect a function to a DynamoDB table
        - How to create a serverless API
        - How to debug and test a serverless function
        - How to organize and grow a serverless application
Materials referred to in the workshop:,HYgVepLIpfxrK4EQNclQ9w
DynamoDB blog Alex DeBrie:
Excellent book for the DynamoDB:

TypeScript Congress 2022TypeScript Congress 2022
118 min
Crash Course into TypeScript for content from headless CMS
In this workshop, I’ll first show you how to create a new project in a headless CMS, fill it with data, and use the content in your project. Then, we’ll spend the rest of time in code, we will:
- Generate strongly typed models and structure for the fetched content.
- Use the content in components
- Resolve content from rich text fields into React components
- Touch on deployment pipelines and possibilities for discovering content-related issues before hitting production