Platform-powered: Building a Frontend Platform to Scale as Fast as You Do 🚀

Rate this content
Bookmark

In 2019, our frontend engineers were asking hard questions about the future of our frontend build system. As new engineering teams continued to spin up new frontend microservices, our custom-built platform grew increasingly fragmented. This led to more headaches for our teams, who were struggling to keep up with maintenance patches and security updates. What could we do to get ourselves out of this mess?


Come join us as we discuss how we built a new platform with Next.js at its core to solve the challenges we faced. With the momentum of the Next.js community, we were able to extend this framework with a unique plugin architecture that allowed our engineers to hot-swap new packages and tools while allowing us to upgrade entire swaths of the stack at once. At the same time, we've multiplied the productivity (and happiness) of every frontend engineer at Lyft. We'll end with some great things to share that can be taken with you to address the challenges you face scaling your own frontend platforms!

Andrew Hao
Andrew Hao
34 min
14 May, 2021

Comments

Sign in or register to post your comment.

Video Summary and Transcription

Lyft has developed a next-generation microservice platform called Lyft Service, built on Next.js. They use a plugin system to reuse code and keep their stack modern. The plugin system allows for easy packaging and delivery of application functionality. Lyft has a structured approach to migrations and versioning, ensuring seamless upgrades. The success of the platform is attributed to organizational and cultural processes, as well as buy-in from senior leadership.

Available in Español

1. Introduction to Lyft's Microservice Platform

Short description:

Today, I wanted to share with you some really great things that the Lyft team has been working on in order to create and scale our next generation micro service front-end platform. Have you ever felt the growing pains of being at a company? You've now got to scale the company and the team to new heights. But however, you're now at a point where it's impossible to manage all these different architectures and different services at scale. What do you do at this point?

Hi, everybody. My name's Andrew. I'm an engineer here at Lyft. Today, I wanted to share with you some really great things that the Lyft team has been working on in order to create and scale our next generation micro service front-end platform.

But before we do that, I want to tell a story to see if you relate. Have you ever felt the growing pains of being at a company? Perhaps you were a small startup that's really growing fast and with newfound success, congratulations, you've now got to scale the company and the team to new heights. And so now, you've gotten new teammates, new teams trying to accomplish and solve all these different business problems, and they're all constrained by being in the same monolithic application.

So of course, you empower those teams, and you split them off, and you send them off to do their thing in a distributed microservices way. Every team is now empowered to build a platform and to work on their own problems as they see fit. And this might succeed for a long time. But however, you're now at a point where it's impossible to manage all these different architectures and different services at scale. Security updates are hard to apply. Platforms start to exhibit some sort of drift, it's difficult to make sure that everyone's following the same conventions. And developers find it very difficult to get things done because they're all looking for support in a very specific way. What do you do at this point?

2. Lyft's Inflection Points and Technical Leverage

Short description:

Most tools and processes support one order of magnitude of growth before becoming ineffective. Software systems are designed for a certain sweet spot of inputs. Changing the amount of input or volume requires thinking about a different architecture. Lyft faced similar inflection points throughout its history. In the early days, monolithic web applications worked. As the business scaled, a generation 2 service based on Node and React was created. However, long-lived services require upkeep and updating applications becomes difficult at scale. The concept of technical leverage is the ability to multiply inputs into outsized outputs. It's important to have a structured way to think about technical leverage. Lyft centralized the bottom layer of its application infrastructure to empower developers.

Will Larsen says, most tools and processes really only support about one order of magnitude of growth before becoming ineffective. Now, that's not to say that if your architecture isn't able to scale two orders of magnitude of growth, it's badly designed. No, all it means is that your software systems are designed for a certain sweet spot of inputs. And once you change the amount of input or volume or throughput that it must handle, you need to actually start thinking about a different architecture altogether.

And let's face a very similar set of inflection points throughout its history as a company. At the first inflection point was probably the early days of Lyft. In the early days were in monolithic web applications, and for the most part, they worked. Now, most of our front-end code back in those days was in a Python Angular monolith, one or two of them, and then as the business scaled, we needed to create and empower developers. So there was a generation 2 service based on Node and React, and those could be cranked out really quickly with an application service generator, which is really just a templated repository that pushed out an application, which has some pretty sane defaults for how we did things under the hood. Well, Lyft also invested in a lot of front-end and general microservice in infrastructure investments, and so we ended up with a microservice explosion. But of course, the problems started catching up with us. Long-lived services, as you know, require upkeep, otherwise they start to decay. If you don't update your application quickly enough, you start to realize that it becomes more and more difficult to move forward to the next required updates. The platform ends up fragmenting if you have enough of these systems at scale. New infrastructure updates are really hard to apply. And not only that, developers find their productivity sapped as they have to keep reapplying new updates after new updates. So, what do we do?

At the very core of what I want to talk about today is the concept of technical leverage. Technical leverage is defined as the ability to multiply your inputs into outsized outputs. So, let's think about the tools we can apply to have outsized impact on developer productivity, reliability, scalability. But where do you start? Very often times when we think about technical leverage or scaling a platform, we, our thoughts automatically go to, hey, let's centralize everything in a repository or an application or service and have that manage that thing. Or we think about, hey, let's go buy this SaaS platform off the shelf that will manage x, y, or z for us. But before we do that, let's actually have a little bit more of a structured way to think about things. So there's a more structured way to think about technical leverage. And the way we think about it is to think about our application as a tiered service. So within our application infrastructure, you can actually imagine that there's the infrastructure layer, the bottom layer, which is corresponds with perhaps the Docker file, or the way that you interface with the cloud service provider, or maybe it's a web pack configuration. In the middle, you have your library layer, which is the third party libraries, you pull off the shelf to get all your things done. And finally, the top layer is the application or the user space code that contains all the code that your engineers write day in and day out to solve the problems in their domain. Now what we want to do is we want to be able to handle the bottom two layers, we want to apply leverage at these layers and empower our users to worry about the top layer. At Lyft, the very bottom layer was initially what we had thought we had solved. We had centralized a web pack build configuration and a lot of our virtualization or containerization things, those things were all managed for our developers.

QnA

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

A Guide to React Rendering Behavior
React Advanced Conference 2022React Advanced Conference 2022
25 min
A Guide to React Rendering Behavior
Top Content
React is a library for "rendering" UI from components, but many users find themselves confused about how React rendering actually works. What do terms like "rendering", "reconciliation", "Fibers", and "committing" actually mean? When do renders happen? How does Context affect rendering, and how do libraries like Redux cause updates? In this talk, we'll clear up the confusion and provide a solid foundation for understanding when, why, and how React renders. We'll look at: - What "rendering" actually is - How React queues renders and the standard rendering behavior - How keys and component types are used in rendering - Techniques for optimizing render performance - How context usage affects rendering behavior| - How external libraries tie into React rendering
Building Better Websites with Remix
React Summit Remote Edition 2021React Summit Remote Edition 2021
33 min
Building Better Websites with Remix
Top Content
Remix is a new web framework from the creators of React Router that helps you build better, faster websites through a solid understanding of web fundamentals. Remix takes care of the heavy lifting like server rendering, code splitting, prefetching, and navigation and leaves you with the fun part: building something awesome!
React Compiler - Understanding Idiomatic React (React Forget)
React Advanced Conference 2023React Advanced Conference 2023
33 min
React Compiler - Understanding Idiomatic React (React Forget)
Top Content
React provides a contract to developers- uphold certain rules, and React can efficiently and correctly update the UI. In this talk we'll explore these rules in depth, understanding the reasoning behind them and how they unlock new directions such as automatic memoization. 
Using useEffect Effectively
React Advanced Conference 2022React Advanced Conference 2022
30 min
Using useEffect Effectively
Top Content
Can useEffect affect your codebase negatively? From fetching data to fighting with imperative APIs, side effects are one of the biggest sources of frustration in web app development. And let’s be honest, putting everything in useEffect hooks doesn’t help much. In this talk, we'll demystify the useEffect hook and get a better understanding of when (and when not) to use it, as well as discover how declarative effects can make effect management more maintainable in even the most complex React apps.
Routing in React 18 and Beyond
React Summit 2022React Summit 2022
20 min
Routing in React 18 and Beyond
Top Content
Concurrent React and Server Components are changing the way we think about routing, rendering, and fetching in web applications. Next.js recently shared part of its vision to help developers adopt these new React features and take advantage of the benefits they unlock.In this talk, we’ll explore the past, present and future of routing in front-end applications and discuss how new features in React and Next.js can help us architect more performant and feature-rich applications.
(Easier) Interactive Data Visualization in React
React Advanced Conference 2021React Advanced Conference 2021
27 min
(Easier) Interactive Data Visualization in React
Top Content
If you’re building a dashboard, analytics platform, or any web app where you need to give your users insight into their data, you need beautiful, custom, interactive data visualizations in your React app. But building visualizations hand with a low-level library like D3 can be a huge headache, involving lots of wheel-reinventing. In this talk, we’ll see how data viz development can get so much easier thanks to tools like Plot, a high-level dataviz library for quick & easy charting, and Observable, a reactive dataviz prototyping environment, both from the creator of D3. Through live coding examples we’ll explore how React refs let us delegate DOM manipulation for our data visualizations, and how Observable’s embedding functionality lets us easily repurpose community-built visualizations for our own data & use cases. By the end of this talk we’ll know how to get a beautiful, customized, interactive data visualization into our apps with a fraction of the time & effort!

Workshops on related topic

React Performance Debugging Masterclass
React Summit 2023React Summit 2023
170 min
React Performance Debugging Masterclass
Top Content
Featured WorkshopFree
Ivan Akulov
Ivan Akulov
Ivan’s first attempts at performance debugging were chaotic. He would see a slow interaction, try a random optimization, see that it didn't help, and keep trying other optimizations until he found the right one (or gave up).
Back then, Ivan didn’t know how to use performance devtools well. He would do a recording in Chrome DevTools or React Profiler, poke around it, try clicking random things, and then close it in frustration a few minutes later. Now, Ivan knows exactly where and what to look for. And in this workshop, Ivan will teach you that too.
Here’s how this is going to work. We’ll take a slow app → debug it (using tools like Chrome DevTools, React Profiler, and why-did-you-render) → pinpoint the bottleneck → and then repeat, several times more. We won’t talk about the solutions (in 90% of the cases, it’s just the ol’ regular useMemo() or memo()). But we’ll talk about everything that comes before – and learn how to analyze any React performance problem, step by step.
(Note: This workshop is best suited for engineers who are already familiar with how useMemo() and memo() work – but want to get better at using the performance tools around React. Also, we’ll be covering interaction performance, not load speed, so you won’t hear a word about Lighthouse 🤐)
Concurrent Rendering Adventures in React 18
React Advanced Conference 2021React Advanced Conference 2021
132 min
Concurrent Rendering Adventures in React 18
Top Content
Featured WorkshopFree
Maurice de Beijer
Maurice de Beijer
With the release of React 18 we finally get the long awaited concurrent rendering. But how is that going to affect your application? What are the benefits of concurrent rendering in React? What do you need to do to switch to concurrent rendering when you upgrade to React 18? And what if you don’t want or can’t use concurrent rendering yet?

There are some behavior changes you need to be aware of! In this workshop we will cover all of those subjects and more.

Join me with your laptop in this interactive workshop. You will see how easy it is to switch to concurrent rendering in your React application. You will learn all about concurrent rendering, SuspenseList, the startTransition API and more.
React Hooks Tips Only the Pros Know
React Summit Remote Edition 2021React Summit Remote Edition 2021
177 min
React Hooks Tips Only the Pros Know
Top Content
Featured Workshop
Maurice de Beijer
Maurice de Beijer
The addition of the hooks API to React was quite a major change. Before hooks most components had to be class based. Now, with hooks, these are often much simpler functional components. Hooks can be really simple to use. Almost deceptively simple. Because there are still plenty of ways you can mess up with hooks. And it often turns out there are many ways where you can improve your components a better understanding of how each React hook can be used.You will learn all about the pros and cons of the various hooks. You will learn when to use useState() versus useReducer(). We will look at using useContext() efficiently. You will see when to use useLayoutEffect() and when useEffect() is better.
React, TypeScript, and TDD
React Advanced Conference 2021React Advanced Conference 2021
174 min
React, TypeScript, and TDD
Top Content
Featured WorkshopFree
Paul Everitt
Paul Everitt
ReactJS is wildly popular and thus wildly supported. TypeScript is increasingly popular, and thus increasingly supported.

The two together? Not as much. Given that they both change quickly, it's hard to find accurate learning materials.

React+TypeScript, with JetBrains IDEs? That three-part combination is the topic of this series. We'll show a little about a lot. Meaning, the key steps to getting productive, in the IDE, for React projects using TypeScript. Along the way we'll show test-driven development and emphasize tips-and-tricks in the IDE.
Web3 Workshop - Building Your First Dapp
React Advanced Conference 2021React Advanced Conference 2021
145 min
Web3 Workshop - Building Your First Dapp
Top Content
Featured WorkshopFree
Nader Dabit
Nader Dabit
In this workshop, you'll learn how to build your first full stack dapp on the Ethereum blockchain, reading and writing data to the network, and connecting a front end application to the contract you've deployed. By the end of the workshop, you'll understand how to set up a full stack development environment, run a local node, and interact with any smart contract using React, HardHat, and Ethers.js.
Designing Effective Tests With React Testing Library
React Summit 2023React Summit 2023
151 min
Designing Effective Tests With React Testing Library
Top Content
Featured Workshop
Josh Justice
Josh Justice
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents- The different kinds of React application tests, and where component tests fit in- A mental model for thinking about the inputs and outputs of the components you test- Options for selecting DOM elements to verify and interact with them- The value of mocks and why they shouldn’t be avoided- The challenges with asynchrony in RTL tests and how to handle them
Prerequisites- Familiarity with building applications with React- Basic experience writing automated tests with Jest or another unit testing framework- You do not need any experience with React Testing Library- Machine setup: Node LTS, Yarn