The Rise of Modern Transactional Stack

Hi, everyone. My name is Yoko. I'm a tech cartoonist, developer, product manager, and lastly partner at H16z investing in the JavaScript ecosystem and developer tools. So today I would like to share a few interesting architectural shifts I have seen after talking to hundreds of developers and tech companies. I'm putting the architectures in the context of transactions since they're so core to our daily lives. They power things from plane tickets building to getting food deliveries to our doors to booking cat sitters and dog sitters. And we're seeing a major shift in how developers use and reason about technologies here.

Well, spoiler alert, JavaScript always wins. So let's talk about why and how. Now, here is the architecture you will see in the old world. As you can see here, there are a bunch of microservices divided between client and the server because they're retaining different languages, also because front end and back end developers can't possibly agree with each other. So when you see the request path, when a request comes in, for example, placing an order. It gets authenticated and then sent to an API. The API endpoint will need to orchestrate hundreds of microservices to deliver a business transaction to the user. And then it will go through things like inventory management service, order processing service, payment processing service. This is when developers use queues and caches as you can see on the lower half of the picture to store what we call application states. Things like order received, order paid. The developers will also hand code the retry logic to get the data to the databases and cross their fingers and hope for the best. We found that as the system scales so does the complexity of managing queues and caches and corner cases. And now we actually see many companies start to build their own new projects very differently.

Now enter the modern transactional stack where you no longer need to provision and manage hundreds and thousands of microservices. Instead, we have got to a point that serverless and JavaScript alone is good enough to power the whole application. So you may be wondering what is the modern transactional stack? First, it's entirely JavaScript and backend is taken care of by a vendor. Sometimes the vendor is serverless. There's no longer what we call frontend and backend. Everyone is a full stack engineer and there are no longer APIs since it's written in the same programming language. And then, now you have things like server components and TRPC just calling a function. We are seeing more and more companies embracing this pattern as they go. What we traditionally call the backend, now we have platforms like Superbase, Convex, Upsash and others that can entirely eliminate the need for the app developers to directly manage backend infrastructure. Trust me, you don't want to work on AWS.

I used to provision infrastructure all the time and it's not fun. In this architecture, you will also utilize services that can scale down to 0 and scale up based on demand quickly. Removing the need for DevOps teams altogether. Upsash is a great example of this.

Second, another trait of the modern transactional stack is that it's powered by component-driven toolings. For authentication, you no longer need to deal with lower-level tools backend engineers traditionally use. Now you can just pick up tools like Clerc and use a signing React component. Another good example here is an open-source project called React Email. You no longer have to code vanilla HTML for transactional emails and now you can just use React and Tailwind directly in your email. For object storage, this is always something that a lot of engineers tell me and even myself, how to learn AWS for. But now you can use something called UploadThing natively in React instead of using AWS S3 directly.

Lastly, one highlight of this stack is that as businesses use more APIs, more functions and accept more webhooks, they essentially have to deal with async functions at scale. So when this happens, there needs to be a central place to orchestrate these things since handcrafted retries are hard. Now we start to see workflow engines playing a central role in the serverless native world. And the way to understand them is that they are the orchestrator of async functions. Here's the intuition. If your app handles hundreds of user signups and then sends a bunch of welcome emails to the users, you can use tools like ingest to write a few lines of TypeScript code to define what functions should be triggered when a signup event happens. It will also retry this part of the logic automatically with exponential backoff if that process fails. And on the client side, if you want, you can use the authoring layer of the state machine called xState. I hook it up with the workflow engine. It's a super cool project, makes things much easier to reason about compared to Redux which I used to use all the time.

There is probably hours and days of things to talk about in a stack we see over and over again here in the serverless era. If you want to chat more, feel free to send me a DM on Twitter at staff Yoko Draws. And that's all I have today. Thank you.