Platform-powered: Building a Frontend Platform to Scale as Fast as You Do 🚀

Hi, everybody. My name's Andrew. I'm an engineer here at Lyft. Today, I wanted to share with you some really great things that the Lyft team has been working on in order to create and scale our next generation micro service front-end platform.

But before we do that, I want to tell a story to see if you relate. Have you ever felt the growing pains of being at a company? Perhaps you were a small startup that's really growing fast and with newfound success, congratulations, you've now got to scale the company and the team to new heights. And so now, you've gotten new teammates, new teams trying to accomplish and solve all these different business problems, and they're all constrained by being in the same monolithic application.

So of course, you empower those teams, and you split them off, and you send them off to do their thing in a distributed microservices way. Every team is now empowered to build a platform and to work on their own problems as they see fit. And this might succeed for a long time. But however, you're now at a point where it's impossible to manage all these different architectures and different services at scale. Security updates are hard to apply. Platforms start to exhibit some sort of drift, it's difficult to make sure that everyone's following the same conventions. And developers find it very difficult to get things done because they're all looking for support in a very specific way. What do you do at this point?

Will Larsen says, most tools and processes really only support about one order of magnitude of growth before becoming ineffective. Now, that's not to say that if your architecture isn't able to scale two orders of magnitude of growth, it's badly designed. No, all it means is that your software systems are designed for a certain sweet spot of inputs. And once you change the amount of input or volume or throughput that it must handle, you need to actually start thinking about a different architecture altogether.

And let's face a very similar set of inflection points throughout its history as a company. At the first inflection point was probably the early days of Lyft. In the early days were in monolithic web applications, and for the most part, they worked. Now, most of our front-end code back in those days was in a Python Angular monolith, one or two of them, and then as the business scaled, we needed to create and empower developers. So there was a generation 2 service based on Node and React, and those could be cranked out really quickly with an application service generator, which is really just a templated repository that pushed out an application, which has some pretty sane defaults for how we did things under the hood. Well, Lyft also invested in a lot of front-end and general microservice in infrastructure investments, and so we ended up with a microservice explosion. But of course, the problems started catching up with us. Long-lived services, as you know, require upkeep, otherwise they start to decay. If you don't update your application quickly enough, you start to realize that it becomes more and more difficult to move forward to the next required updates. The platform ends up fragmenting if you have enough of these systems at scale. New infrastructure updates are really hard to apply. And not only that, developers find their productivity sapped as they have to keep reapplying new updates after new updates. So, what do we do?

At the very core of what I want to talk about today is the concept of technical leverage. Technical leverage is defined as the ability to multiply your inputs into outsized outputs. So, let's think about the tools we can apply to have outsized impact on developer productivity, reliability, scalability. But where do you start? Very often times when we think about technical leverage or scaling a platform, we, our thoughts automatically go to, hey, let's centralize everything in a repository or an application or service and have that manage that thing. Or we think about, hey, let's go buy this SaaS platform off the shelf that will manage x, y, or z for us. But before we do that, let's actually have a little bit more of a structured way to think about things. So there's a more structured way to think about technical leverage. And the way we think about it is to think about our application as a tiered service. So within our application infrastructure, you can actually imagine that there's the infrastructure layer, the bottom layer, which is corresponds with perhaps the Docker file, or the way that you interface with the cloud service provider, or maybe it's a web pack configuration. In the middle, you have your library layer, which is the third party libraries, you pull off the shelf to get all your things done. And finally, the top layer is the application or the user space code that contains all the code that your engineers write day in and day out to solve the problems in their domain. Now what we want to do is we want to be able to handle the bottom two layers, we want to apply leverage at these layers and empower our users to worry about the top layer. At Lyft, the very bottom layer was initially what we had thought we had solved. We had centralized a web pack build configuration and a lot of our virtualization or containerization things, those things were all managed for our developers.

We had centralized a web pack build configuration and a lot of our virtualization or containerization things, those things were all managed for our developers. But that middle layer libraries weren't very well done. So when we went back to the drawing board, the team went to work. We created a Generation 3 service called Lyft Service. This would be our next generation platform. And these three principles are our guiding principles.

Number one, we stand on the shoulders of giants. Number two, we simplify code in order to understand it. And number three, we standardize so we can automate. Let's go through these one by one.

First, we stand on the shoulders of giants. Our developer platform team evaluated a bunch of candidates for our next generation platform and we landed on Next.js. What we liked was that Next.js had a great opinionated architecture and it was solving so many of the things that we wanted to solve. Things like the build configuration, static site generation, server side rendering, AMP pages, code splitting, dynamic imports. These were all problems that we were already sort of solving, but not yet fully, as engineers at the edges of the organization had done so on one-off sort of instances, but they hadn't reintegrated them back into the system. But when you build on a community-oriented product like Next.js, you get to take on their momentum. And so now, we're actually able to focus on other things rather than having to maintain our own internal architecture.

Second of all, we simplify to understand. So one thing that we got by going to Next.js was the philosophy of choosing convention over configuration. This actually has the effect of simplifying our application by allowing us to delete a lot of boilerplate code. Here's an example. With Next.js applications, we have a file system router. It's one of the most distinct marks of a Next app. And depending on where you put a file in the file system, its path structure mirrors the URL structure of the route that it serves. It's a really powerful idea, and it allows us to delete a lot of our React router code. Additionally, there's conventions around, you know, getting server-side data and passing that over to React. We like the way that they have those things out of the box, and we thought our systems would benefit greatly by landing on those conventions. So in the end, we have lower cognitive load working in these apps, which means our developers are more productive as they switch and context switch between different applications.

Next, we standardize and we automate. We built a plug-in system that standardizes our library integrations, and then we have a migration system that makes migrations a first-class citizen of our new framework.

I'll talk a little bit more about these, but the net effect of having these two new system features is that we have the ability to reuse code, keep the stack modern, and prevent code drift. Let's look at our plugins. So when I'm talking about a plugin, I'm not referring to a Next.js plugin, which is actually a slightly different thing. I'm actually referring to our own internal Lift Service plugin, which is kind of like a framework plugin that allows you to bundle and inject a bunch of custom hooks together. Things like webpack build configurations, express middleware, and Next.js application configurations.

Let's take a look at one to illustrate what I mean. Over here, I've developed a slightly contrived example of a cookie authentication plugin. Let's say that some team in your company has built a cookie auth service that basically, you know, it writes a cookie anytime somebody logs in. So you want to be able to distribute this behavior to other micro front ends or micro services that you have within the organization, so you create a plugin for this. So here's the user experience. They install the plugin package and then they new up an instance of that plugin and pass it into a initialization array. Our service, our framework will take this array and install and inject the hooks at the right place. From the developers point of view, they are now free to use whatever convenience methods you provide in the plugin to your advantage. So over here I've created a hook which provides two items. It provides a user id and it provides an is logged in helper method. These are convenience tools that your developer will be able to use to implement whatever your plugin provides. Notice here that there is no mention of in the internal guts of this plugin. You don't have to know if there's express middleware involved. You don't have to know if there are cookie parsing and what secrets to pass the cookie and all this other stuff. That stuff is abstracted away for you behind the plugin.

Let's actually open the hood now into a plugin to see what's under the hood. So we'll start with looking at an express server hook. This express hook takes in an express app as an argument and allows you to modify it. So over here you'll see we use a cookie parser middleware, a third party package that gives us access to the cookies, and then second of all, we parse those cookies. I've hand-waved away the implementation of that, but the most important thing is that we take that user ID value and we stuff it into the local variable object on the response. We'll use this later on in the next example. Next we have the nextApp hook. This nextApp represents the root node of a React application. And over here there's two things going on. If you're unfamiliar with this pattern in Next, there's a static method called getInitialProps.

GetInitialProps essentially runs before the app renders and it fetches data from the server. The code I provided here will basically run on the server context when that user ID is available and that response object is available. It'll take that user ID and pass it back through to your React app into the props. In the props you'll see that we provide a provider, basically a cookieAuthContext provider, and we provide that user ID for use in the rest of the React app. That's kind of the key bridge between the server world and the React world.

Over here we basically hook everything up into a plugin that when we specify the hooks in which to install and to implement these overrides and then we create our developer-facing conveniences. So over here we have a hook and that hook provides two things. Number one, it it grabs that user ID from the context. And number two, there's a simple helper method that allows you to do something with that. Pretty easy, right?

So what this means is that we've made an easy way to package and deliver application functionality in as little configuration, almost zero configuration, a way possible. So you'll see the power of this, because internally we've got all these plugins going on. We've got plugins for state management, GraphQL, UI components like Live Product Language, which is our internal UI pattern library, material UI style components. We have authentication and authorization plugins, internationalization, performance tracking, feature flagging, MirageJS, and other testing utilities, and logging, metrics, and bug reporting. Lots of things. We've even got really big plans to provide increased developer support tools. Our developer platform team wants to make it even easier for folks to be able to report bugs, grab stack traces, and ask for help. The really great thing about this plugin system is it's created a flywheel effect. Users are contributing back to these plugins. Our developers, our developer engineers, at the edges, they're solving problems in their systems. They're saying, hey, I'm going to build my implementation as a plugin and contribute it back to the ecosystem. Now, their impact is multiplied across all of our developers, across the organization. Over 40% of our plugins have been product engineer contributions.

Now, let's turn our attention to our migration system. Migrations are the secret sauce in how we plan to prevent or work against the effects of architecture drift. They're guardrails. Essentially, migrations are JS code shift scripts. JS code shift, if you're unfamiliar, is a framework that helps you modify JavaScript and rewrite it from one form to another. I won't go into the details on how a JS code shift code mod works. You can easily look it up online. However, I think it will be appropriate for me to illustrate what it looks like from the outside.

So, here's an example. A developer recently changed one of our logging plugins. In the old world, it would export an instance of a logger, and now it exports a factory function. So, instead of shipping a breaking version of this library and maybe writing in the changelog or the readme, hey, if you have it this way, please update all references to use the new way, we automate that under the hood so a script is provided to update all instances of the old world to the new world. Therefore, all breaking changes theoretically aren't really breaking anymore. Theoretically, everything should just work as you migrate upward.

One more key thing to note is that our migrations are versioned. This means that every migration corresponds with a specific package or platform version that it was introduced at. We store the state of the latest migration that you run. If you run a migration for a certain package at a certain point, we know that the next time you upgrade, we only need to run the next upgrade for the version ahead of you. If you skip or update multiple versions, we'll know to sequentially run each of those migrations, which should theoretically end you up at the latest state with as little intervention as possible.

One very bold move, or bold design decision, in my opinion, that the team made was they decided to pin the plugin and the platform versions together, so that means they all move together at the same time. And this was inspired by the Babel and the React communities, where they do much of the same thing. The extensions that make the platform work are pinned or moved together with the main library. And so what this means is if you want the latest version of the platform, you've got to also upgrade all your plugins to the latest version. Or if you want the latest version of the plugin, you've got to do it vice versa. You've got to upgrade your platform. And with migrations at a at the root of of the system empowering all these migrations, empowering all these upgrades, upgrades now become a very seamless and simple thing to do. By doing this, we believe we've designed the right incentives for developers to upgrade to the latest. And structurally, structurally speaking, the the platform is designed to always stay at the latest version.

Finally, you can't build a system like this and just throw it over the wall. A lot of organizational and cultural processes have to come together to make this thing succeed. So, one of the very first things that the team did was they held hands-on migration workshops, where teams that had systems to be migrated would meet with an engineer or two from the platform team and they would handhold them to teach them about the ins and outs of the new platform. And when that was done, then the team would develop an insider's knowledge about how this thing worked and they'd be able to teach other people on their teams as well. So, they're able to organically create and share that knowledge. This also gave our engineers the ability to know ahead of time, you know, with a few guinea pig teams, like what would go wrong so they can go back and tweak the system before expanding their radius.

Another thing was that migration scripts, you know, code mods, were written to migrate the bulk of generation two systems to generation three systems. So, this also made the ability to migrate much easier and it also took down the kind of the intimidation factor of migrating from gen two to gen three. There were lots of opportunities to do some internal evangelism at brown bags and all hands meetings. And finally, it can't be overstated enough how important it was to get buy-in from senior leadership. Who were sold on the productivity wins that this would give our entire organization and supported it with program management that made sure that this migration was successful. Today, over 60% of our applications are now on our generation three lift service and more to come. We've seen high adoption of this new platform precisely because we've generated enough buzz and developer productivity that people are telling, singing the praises of the new system to other developers. Not only that, because we've seen evidence of such quick adoption, we're convinced that this quick adoption will continue as we continue to roll and ship out new releases of this platform.

So, what do you do, what can you take from these lessons? As you're thinking about how to apply these to your systems, we want to ask the questions. Number one, how are you standing on the shoulders of giants? How are you moving forward with the community? How can you take part in the momentum of open source so that you can also leverage their momentum? Number two, how are you simplifying your code? How are you making sure that boilerplate or unnecessary code can be simplified down to its bare essence? For us, it was choosing to opt for convention over configuration. What does that look like for you? And number three, we standardize and automate. So, where are places where you're seeing divergence? How can you bring that divergence back into one place by perhaps creating an abstraction layer or centralizing and platformizing? And finally, how will you use automation to multiply your impact to make sure that as you upgrade, nothing gets left behind?

Well, that's it. That's what we wanted to share with you today. We hope to see you in the Q&A and thank you for attending this talk at React Summit. Thank you.

Thank you so much for that amazing talk, Andrew. I've learned a lot. I hope you have too as well. Let's go and check on that question that Andrew asked us, asking how is your organization scaling your front-end stack? I'm going to invite Andrew on so we can talk about the results. So, if we look, we see that 58% that say that their organizations are centralizing libraries and services, followed by spinning up a platform team at 17%, and leveraging open source or other platforms at 15%. Was that kind of what you expected? Yeah, that is very much what we've seen, as we've also talked to some other teams as well. It seems like one of the first things that people want to do is, yeah, you want to pull stuff out into libraries and services and things like that, stuff that makes sense and stuff that overlaps. It's in our DNA as developers to dry everything up. So let's pull everything out together. And I think the next thing is actually really important, building a platform team. When you get a certain point in time, you need to dedicate specific people, actual humans, full-time, making sure your ecosystem and your tools are really sharp.

I absolutely agree, totally great answers. I love those three points of standing on the shoulders of giants, simplifying and standardizing, those really resonate with me. But we've got a couple of questions from the Discord, I'm just going to go through some of them.

So one from Qabs says, are any of these tools open source and do you have any write-ups on the plugin architecture setup? Thanks for asking. So we do get this question a lot, hey, are you going to open source this? The answer I can give you is that there have been discussions internally, there are no current plans to open source this framework. However, as I've shown here in this talk, the setup of such a platform is not too difficult to do. And please feel free to get in touch either on Discord or on Twitter, we can connect to you or set up a chat to see how we can, you know, create some more ideas.

Cool. I mean, those of you keep hitting them up on Twitter, maybe they will open source it if enough people get behind the idea. We've got another question from Vasili saying, that's a lot of plugins, how do developers decide what's worthy of being a plugin? Well, that is the really awesome thing about how the team has designed the plugin system. The plugin system is anything that solves your problem, right? Like, at a certain size, like, really, you cannot just top down say, like, use this plugin, don't use that plugin, use this one. I mean, you probably can, if there's like, security problems or like, specific things you have to enforce, right? But in the end, like every developer at, like what I call the edge, right? They're out there building and solving problems in their area of business, right? So they are going to be the ones to know best what tools they need, right? We've got developers who picked up React query or something like that, right? And that might have solved a really great problem for them in their area of the org. No one's going to tell them, hey, like, actually, you should be using this plugin instead, right? It's a very much like, hey, I found a tool that works for me. I want to actually bring it back, you know? And I made a plugin, and I want, you know, and I'm going to tell other people about it. So it's very organic process. Another example of this is, you know, the usage of, I remember one of our, one of our developer product engineers brought in MirageJS because it solved the problem they had locally, you know, and that actually solved like, you know, other teams came out of the woodwork saying, hey, this is awesome. This solves something that we've been running up against as well. Like you have to understand at a certain organization size, like it's hard for developers to find each other, right? And you're nodding your head because I'm sure you felt the same way too. And when I first joined, you know, most of my career, I've worked at companies of maybe 200 people or less, you know, maybe a 50 person engineering team, everyone knows everyone else, right? But now I'm in a company with, you know, over a thousand engineers. It's very difficult to know what everybody's doing all the time, right? So, that's why having these like tools and the ability to extend your architecture, but to do it in a safe way and to allow it to scale is so important.

Definitely. I love that bit about finding people. The number of times I've gone through Slack threads, trying to find the owner of something. It brings back dark memories. Another question we've got from Bear Man saying, are there any headaches or have you evolved any rules of thumb around versioning your internal plugins across the microservices? Thank you for asking that question. Obviously, it's not all like daisies and roses, right? The challenge is, I think, with versioning, right, like is, are the universal problems of versioning, right? When you bind things together, right, like when you say like, hey, like this plugin must work with this version, you're going to have some incompatibilities because some people are not always going to be at the latest versions, right? There's going to be some debugging problems, right, when our platform team has to jump in and help another team and realize, hey, actually what you're using is this older version that had this edge case bug, you know, and it took a long time to find that bug, right? And that's kind of why we've developed this migration story. Like we want to make it as easy and as frictionless as possible to get your system moved up to the latest version, right, like that should be the easiest thing ever to do, run a few commands and boom, you're there. In practice, it is more, it obviously is more difficult. Everyone's application is going to have some strange part of its surface area that's going to require a little bit of manual work, and so I would say a lot of our issues are also around the cross compatibility of systems. So you'll have plugins that depend on other plugins, and sometimes those plugins have certain assumptions about how the world works that you have to like hammer out, you know, and so that's another reason why we kind of pin the versions of all the plugins together, so that we can guarantee and like put a stamp of approval like this version is guaranteed to work across other versions of the same platform.

That definitely makes sense. That's a good answer. We have another question, because you're serving so many different developers from so many different teams. This one comes from the Poona 777. How do you handle different requirements coming from different product teams? I'd like to dig in a little bit more. I mean, I can kind of understand a little bit about what you're asking.

First of all, the developer platform, we call them front-end experience team here. I wish I had them here with me too, so they could represent themselves better. They get to kind of define the architecture of this platform. By doing a lot of listening tours with different engineers, they are on the ground listening to or working with our product teams. Just by working with your customers, to the platform team, product engineers and their services are their customers. They have an idea for what's working or what's broken and how to solve the problems for different product engineers.

In that sense, nobody is telling the platform team how to build the system. The platform team, based on the problems they hear, think to themselves like, hey, here's a great idea. I want to take it forward. That's how the leads of the platform team developed this plugin system because they felt the pain. They understood the pain from having to jump back and forth through all these different services and realize, hey, everything's out of date. Everything needs to be upgraded. How do we solve that problem, both for this product team but also for ourselves, and lower that support burden on our team? Yeah. One thing you said about listening and how they're the customers, oftentimes we think of the customers as the people, the end users of the app. But oftentimes they're people whose roles, their customers are the other developers. And learning what those people actually need helps you to inform your design choices.

Another one from Vasily Shelkov says, can you give examples of what Lyft has done to show how much working on these can improve productivity? E.g. spent time spent on x is reduced delivering FOSTA by x. I don't have those numbers right off of that. They are definitely quantified somewhere. In terms of, like, support load or productivity. We did do some kind of raw measurements of the new stack versus the old stack in terms of app boot up speed, lines of code removed and all that stuff. We do have a blog post on our engineering blog, eng.liff.com, and you will be able to find it where we do quantify some metrics and some wins from this platform.

As for, like, overall, like, developer productivity, I do say, like, developers are more productive. I don't have numbers to share, like, exactly how much more productive. But that's a great exercise. We would love to share that if we are able to. Nice, nice. Another question from Min who says, do you have a lot of Next.js apps? And if so, how do you aggregate them all into one app? That's a great question. So, first of all, yes. Lift has lots of Next.js apps. At this point, Lift has over 100 front-end services, over 60 of which are Next.js apps. So, yes, we got lots. We don't actually try to merge them into one application. Although, you can. And I do know there's a Vercel platform feature to allow you to run sub-apps inside like parent apps. We do not do that, especially for the purposes of decoupling teams. So, I would definitely refer you to the Next.js core team or to that community to see if you do want to merge applications into a single Next.js app, but that's not something that we do at Lyft.

Cool. There's so many questions. One more question that I'm going to point out to you is how does the platform team do user research? Because remember you spoke about going and actually listening to those developers. Do you have your own user researchers in-house to understand them? And how do you not just buy us to sort of what the platform team prefer themselves? That's a great question. I would say the team themselves are doing the research. Because they're on the ground troubleshooting, they're actually going out and talking to engineers and getting a sense for the pain point. So kind of our lead architect walked me around the block. We went on the coffee walk pre-COVID, and he just kind of shared a little bit about what was on his mind and kind of like testing whether, hey, would this platform solve your problems? Here, talk this through with me. So there was a lot of input in the design phase of this new platform. I would also say that user research, yeah, in many ways, it's all ground level, you know. It would be awesome to have an embedded UX researcher kind of like thinking about the developer experience. I think that would be super awesome. Cool. Well, thank you so much, Andrew, for answering all those questions. I know there were a few more that we missed, but you folks can speak to Andrew yourselves over in the spatial chat. So Andrew is going to head off from here and chat to you there.