Generating Typed Code

Rate this content
Bookmark

For the past six at Rollbar the engineering team has been working tirelessly to migrate our legacy application to a modern Javascript tech stack. We’ve leveraged Next.js, GraphQL and a number of other JS tools to ship more features, make our dev team happier and achieve an impressive 7-fold reduction in our p99 app performance. I invite you to join me as I present the various techniques and architecture we choose to use to quickly migrate away from our legacy monolith.

22 min
21 Sep, 2023

Comments

Sign in or register to post your comment.

Video Summary and Transcription

Today, I'm going to be talking about how type generation helped reduce bugs and improve our developer experience at Rollbar. The Monolith tech stack is built primarily on Python Pyramid, a web framework that's similar to Django, Ruby on Rails, or Express. To address debugging, performance, and code navigation challenges, we migrated to a tech stack that includes TypeScript, Next.js, GraphQL, and React Query. Codegen is used to generate TypeScript code on both the server and client side, enhancing debugging, performance, and code navigation.

Available in Español

1. Introduction to Rollbar Monolith and Tech Stack

Short description:

Today, I'm going to be talking about how type generation helped reduce bugs and improve our developer experience at Rollbar. I'll provide an overview of the architecture that we decided to implement. The Monolith tech stack is built primarily on Python Pyramid, a web framework that's similar to Django, Ruby on Rails, or Express.

Hello, everyone. Today, I'm going to be talking about how type generation helped reduce bugs and improve our developer experience at Rollbar. I'm Paul Sereno. I'm a senior software engineer at Rollbar. I have about 10 years of experience as a software developer, working on web-based applications, and I live in Austin, Texas.

For today's agenda, I'm going to first give a brief introduction into the Rollbar monolith. I'll talk about some of the technical challenges we experienced while maintaining the monolith code. And I'll explain why we made the decision to migrate our code base to TypeScript. Then I'm going to talk about the migration strategy into TypeScript, and I'll provide an overview of the architecture that we decided to implement. After that, I'll discuss how we were using GraphQL Code Gen at Rollbar, and I'll provide a few code examples that will demonstrate various ways we're using Code Gen on both the server side and client side of our application.

Okay, first let's go through the Monolith application. The Monolith tech stack is built primarily on Python Pyramid, a web framework that's similar to Django, Ruby on Rails, or Express. We're using a number of different database technologies like MySQL, Elasticsearch, and Clickhouse. The command is written in React 16, so there's no hooks and everything is styled using a mix of Sass and CSS.

2. Migrating to New Tech Stack

Short description:

Throughout the years, the Monolith has had 78 contributors, over 44,000 commits, and over 9,000 pull requests. Developing and deploying in a monolithic code base has its benefits, such as easier development and deployment processes. However, debugging, performance, and code navigation have been significant challenges. To address these issues, we decided to migrate to a newer tech stack that includes TypeScript, Next.js, GraphQL, and React Query. This tech stack allows us to leverage type safety, flexibility in backend migrations, and efficient API calls through React Query.

Throughout the years, the Monolith has had 78 contributors, over 44,000 commits, and over 9,000 pull requests, and is 11 years old. There are some benefits of working in a monolithic code base. It's much easier to develop in a monolithic code base, because there's often boilerplate code that developers can copy and paste and tweak to get something working faster. Developing a monolith application is sometimes easier if you can fit everything into a... Sorry about that.

Deploying a monolith application is sometimes easier because you can fit everything into a single Dockerfile. And code search is often a better experience in monolithic applications because a line of code that you'd like to search for is all in a single repository. So it's much easier to search for single lines of code.

Some of the challenges that we experience with working in our monolith applications have been debugging has been a big challenge because there's a lot of code that you have to read through to fully understand the root cause of certain issues. We use Rollbar ourselves to triage issues, and while the grouping features that Rollbar provides are nice, we still find it difficult to root cause some parts of the code base that are really complex. Performance is another issue with our monolithic code base, and it's mostly due to inefficient code. For example, things like making duplicate database calls and doing heavy post-processing on fetching data has been a pretty big bottleneck. We've also experienced issues with just navigating through code. It could sometimes take hours to fully understand how some things work. And given these issues, we decided to make the decision to migrate away from working in a monolithic application towards a newer tech stack.

And that tech stack includes TypeScript. So the new tech stack leverages Next.js, GraphQL, and React Query to help resolve some of the issues related to our monolith application. And of course, everything's written in TypeScript. The new tech stack that we're in the process of migrating to still uses parts of the monolithic application. We made those features from the monolith available through a number of internal API end points. The GraphQL server is essentially used as an API proxy that sits between our internal API and our front end application. And we made the decision to use GraphQL in this way because it not only provides TypeScript, but it not only provides type safety between the server and the client, but it also gives us flexibility in terms of back end migrations. For example, we're planning on placing some of the internal API end points with gRPC calls. In this proxy, architecture allows us to experiment with different back end solutions. All of our API calls from the front end are made through React Query. And the React Query client that we're using is completely generated from the GraphQL queries themselves. And I'll talk about that in a minute. But this setup has allowed us to get a lot of type safety into the front end. So I mentioned that we generate our front end client. So specifically we use a tool called GraphQL Codegen.

3. Using Codegen for Type Generation

Short description:

We use Codegen to generate TypeScript code on both the server and client side. It provides an easy-to-use command line tool. For resolver functions, we manually add types to ensure type safety. Codegen allows configuration overrides for type generation. On the client side, Codegen can generate type-safe hooks from GraphQL queries. Migrating to this architecture improves debugging, performance, and code navigation. TypeScript enhances developer experience by enabling parallel work.

And this is an open source tool that will automatically generate TypeScript code from either a GraphQL schema or a GraphQL query operation. So we use Codegen on both the server and client side. On the server side, we use it to generate types for our GraphQL resolvers. And on the client, we use it to generate the React query client from our GraphQL queries.

I'll provide a few examples on the differences between how we use it on the server and client. Using Codegen itself is pretty easy. GraphQL Codegen provides a command line tool that is pretty easy to use. You just have to run GraphQL Codegen and give it a config.

So I just want to give an example of specifically how we're using GraphQL Codegen on the server. This is a code snippet taken from our code base. This represents the environment connection schema that is designed to represent a list of project environments. And using the code connection edge nomenclature is sometimes common in GraphQL when representing paginated lists of data. The resolver for this connection simply fetches from the project environments database and returns a list of environments.

One of the issues that we found with using TypeScript in GraphQL has been adding type safety to these resolver functions. For example, by default, arguments like limit after filter won't get types generated for them. So we have to manually add in types ourselves. So this is an example of a type that we explicitly defined for those resolver arguments. And GraphQL code gen provides a workaround for these issues. It allows you to explicitly add in configuration for overriding their internal generation. So for example, in this case, we are overriding the environment's connection generation using our explicitly defined types here. In the environment connections model, we mapped to the environment connection schema definition.

On the client side, you can use the command line tool the same way by simply calling GraphQL CodeGen. Code generation is a lot easier on the client. So given a GraphQL query such as this one, in this example we are fetching a list of environments from our GraphQL server. And we can generate a hook from that query using GraphQL CodeGen. So this hook will fetch the data that we need, and it will be completely type safe. So in this example, as you can see, when you call, or when you type in through your text editor query.results, you'll be able to see the various attributes that are on that object.

Some of the benefits that we've experienced migrating to this architecture have been debugging. So it's much easier to have the type checker catch any potential issues. Performance, there's various compile time optimizations that you can get from having your code base in TypeScript, and we have seen a performance boost in our application. In terms of navigating the code base, having those type annotations have made working with internal libraries a lot easier to use, especially libraries that don't have very good documentation. And an added benefit to using TypeScript has been it's improved our overall developer experience because it's allowed front-end and back-end developers to work in parallel without having to depend on each other.

Some of my closing thoughts are migrating a monolithic code base is challenging, but with the right architecture, even a small team can make it possible. So we benefited a lot from setting up those internal endpoints and simply using GraphQL as a proxy to our internal API. And overall TypeScript improved our ability to debug and optimize our code. That's all.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Remix Conf Europe 2022Remix Conf Europe 2022
23 min
Scaling Up with Remix and Micro Frontends
Top Content
Do you have a large product built by many teams? Are you struggling to release often? Did your frontend turn into a massive unmaintainable monolith? If, like me, you’ve answered yes to any of those questions, this talk is for you! I’ll show you exactly how you can build a micro frontend architecture with Remix to solve those challenges.
Remix Conf Europe 2022Remix Conf Europe 2022
37 min
Full Stack Components
Top Content
Remix is a web framework that gives you the simple mental model of a Multi-Page App (MPA) but the power and capabilities of a Single-Page App (SPA). One of the big challenges of SPAs is network management resulting in a great deal of indirection and buggy code. This is especially noticeable in application state which Remix completely eliminates, but it's also an issue in individual components that communicate with a single-purpose backend endpoint (like a combobox search for example).
In this talk, Kent will demonstrate how Remix enables you to build complex UI components that are connected to a backend in the simplest and most powerful way you've ever seen. Leaving you time to chill with your family or whatever else you do for fun.
JSNation Live 2021JSNation Live 2021
29 min
Making JavaScript on WebAssembly Fast
Top Content
JavaScript in the browser runs many times faster than it did two decades ago. And that happened because the browser vendors spent that time working on intensive performance optimizations in their JavaScript engines.Because of this optimization work, JavaScript is now running in many places besides the browser. But there are still some environments where the JS engines can’t apply those optimizations in the right way to make things fast.We’re working to solve this, beginning a whole new wave of JavaScript optimization work. We’re improving JavaScript performance for entirely different environments, where different rules apply. And this is possible because of WebAssembly. In this talk, I'll explain how this all works and what's coming next.
React Summit 2023React Summit 2023
24 min
Debugging JS
As developers, we spend much of our time debugging apps - often code we didn't even write. Sadly, few developers have ever been taught how to approach debugging - it's something most of us learn through painful experience.  The good news is you _can_ learn how to debug effectively, and there's several key techniques and tools you can use for debugging JS and React apps.
React Day Berlin 2023React Day Berlin 2023
21 min
React's Most Useful Types
We don't think of React as shipping its own types. But React's types are a core part of the framework - overseen by the React team, and co-ordinated with React's major releases.In this live coding talk, we'll look at all the types you've been missing out on. How do you get the props type from a component? How do you know what ref a component takes? Should you use React.FC? And what's the deal with JSX.Element?You'll walk away with a bunch of exciting ideas to take to your React applications, and hopefully a new appreciation for the wonders of React and TypeScript working together.

Workshops on related topic

React Advanced Conference 2021React Advanced Conference 2021
174 min
React, TypeScript, and TDD
Top Content
Featured WorkshopFree
ReactJS is wildly popular and thus wildly supported. TypeScript is increasingly popular, and thus increasingly supported.

The two together? Not as much. Given that they both change quickly, it's hard to find accurate learning materials.

React+TypeScript, with JetBrains IDEs? That three-part combination is the topic of this series. We'll show a little about a lot. Meaning, the key steps to getting productive, in the IDE, for React projects using TypeScript. Along the way we'll show test-driven development and emphasize tips-and-tricks in the IDE.
React Advanced Conference 2022React Advanced Conference 2022
148 min
Best Practices and Advanced TypeScript Tips for React Developers
Top Content
Featured Workshop
Are you a React developer trying to get the most benefits from TypeScript? Then this is the workshop for you.In this interactive workshop, we will start at the basics and examine the pros and cons of different ways you can declare React components using TypeScript. After that we will move to more advanced concepts where we will go beyond the strict setting of TypeScript. You will learn when to use types like any, unknown and never. We will explore the use of type predicates, guards and exhaustive checking. You will learn about the built-in mapped types as well as how to create your own new type map utilities. And we will start programming in the TypeScript type system using conditional types and type inferring.
React Day Berlin 2022React Day Berlin 2022
86 min
Using CodeMirror to Build a JavaScript Editor with Linting and AutoComplete
Top Content
WorkshopFree
Using a library might seem easy at first glance, but how do you choose the right library? How do you upgrade an existing one? And how do you wade through the documentation to find what you want?
In this workshop, we’ll discuss all these finer points while going through a general example of building a code editor using CodeMirror in React. All while sharing some of the nuances our team learned about using this library and some problems we encountered.
TestJS Summit - January, 2021TestJS Summit - January, 2021
173 min
Testing Web Applications Using Cypress
WorkshopFree
This workshop will teach you the basics of writing useful end-to-end tests using Cypress Test Runner.
We will cover writing tests, covering every application feature, structuring tests, intercepting network requests, and setting up the backend data.
Anyone who knows JavaScript programming language and has NPM installed would be able to follow along.
Node Congress 2024Node Congress 2024
83 min
Deep TypeScript Tips & Tricks
Workshop
TypeScript has a powerful type system with all sorts of fancy features for representing wild and wacky JavaScript states. But the syntax to do so isn't always straightforward, and the error messages aren't always precise in telling you what's wrong. Let's dive into how many of TypeScript's more powerful features really work, what kinds of real-world problems they solve, and how to wrestle the type system into submission so you can write truly excellent TypeScript code.
Node Congress 2023Node Congress 2023
63 min
0 to Auth in an Hour Using NodeJS SDK
WorkshopFree
Passwordless authentication may seem complex, but it is simple to add it to any app using the right tool.
We will enhance a full-stack JS application (Node.JS backend + React frontend) to authenticate users with OAuth (social login) and One Time Passwords (email), including:- User authentication - Managing user interactions, returning session / refresh JWTs- Session management and validation - Storing the session for subsequent client requests, validating / refreshing sessions
At the end of the workshop, we will also touch on another approach to code authentication using frontend Descope Flows (drag-and-drop workflows), while keeping only session validation in the backend. With this, we will also show how easy it is to enable biometrics and other passwordless authentication methods.
Table of contents- A quick intro to core authentication concepts- Coding- Why passwordless matters
Prerequisites- IDE for your choice- Node 18 or higher