React Summit

React Summit 2023

* intro to DB application architecture (gateway + backend, multi-module react app + component library, etc) * Deep dive into CI/CD pipeline to do component library build + publish, app deploys pre canary (branch, dev, staging, prod), Lambda@Edge role * Side  note - multi module applications, enabling the right level of modularity, and B2B SaaS vs B2C * First canary problem - run full FE regression against stable (release candidate) backend early and often - before FE release candidates are cut * Second canary problem - run pointed FE regression even earlier - before MRs are even merged * Third canary question - what does this look like in production? Once again, B2B SaaS vs B2C * End result - Canaries in the CloudFront!

Canaries in the CloudFront

Hi, everybody. Thanks for joining. My name's Vijay Ramesh. I'm a VP of Application Development at Demandbase. And today I wanted to talk to you a little bit about something I like to call canaries in the cloud front. Before we jump in, right, so we only got about 10 minutes today. And that's not a lot of time, but in a way it is. There's a lot of content out there. There's a lot of things to learn, a lot of things to dig into, and not that much time in any given day. So let me cut to the chase first and talk a little bit about what am I going to learn if you spend the next nine and a half minutes here listening to my talk. So number one, I want to talk about what it means to design your front-end CI CD systems in a way that enable your teams to move faster with less risk. And as a part of this, what does it mean to think about your content delivery network as an integral part of your application architecture? And then finally, I want to touch a bit on how the type of business that you run, the type of product you make, the type of customers and market that you serve, how that can help determine what shift left could mean for your front-end teams. And just briefly, like I said, so I'm a VP of Application Development at Demandbase. Over 20 years now, full stack web app type experience, big focus on building rich applications atop of big data and ML platforms. And today at Demandbase, I lead three of our application engineering groups. So before we start, let's actually go maybe one step from before we start. And an architecture diagram like this might be familiar to people who have been doing this for a long time, right? So think in the pre-CDN days, maybe even in the pre-single-page app days, it would be common to have some monolithic system like a Rails app. That app is going to go and be fronted by some sort of load balancer and requests from a web client are going to hit the load balancer, go back to the back-end, and the back-end is going to serve up its own front-end. So let's go maybe one step forward here and introduce a CDN, right? So at Demandbase, we're using AWS, and so we use CloudFront here. And what do we get when we go and introduce a CDN like this? Well, the biggest bang for the buck, I think, is this idea of edge caching. So if Demandbase is building up its React app and storing it off in US East 1, say in Virginia, well, meanwhile, me as a user coming from Berkeley, California, I want to be able to fetch those static assets from a edge cache that's close to my location. And CloudFront enables this pretty natively. So as a user coming in from California, I'm going to go hit the edge cache, say in US West 1. Maybe we've already got the latest build there, and so I don't even need to fall on back down to the origin. Or in the case that I need to, well, the origin is going to go and warm up the cache on the edge, so the next user, say, coming from San Francisco, gets the full application. And already here, right, you need to start thinking about, well, how does introducing this CDN technology impact your CI-CD strategy? In this case, right, you need to make sure that you're going to invalidate the cache as you deploy new builds, such that a user coming, say, from one of these West Coast edges is able to pick up the latest build. And then also, let's talk a little bit about, you know, what are the pre-reqs for the CI-CD system that we started with when we really wanted to shift left? So number one, dev and staging, right, full pre-production environments, and a way locally for a developer to go run the full front-end application, but talking against, say, that staging or that dev backend. Number two, we have a large suite of Selenium integration tests, and importantly, these only really will run in the staging environment due to a general dependency on a stable backend. In the dev environment, backend teams might be testing out feature branches and things like that, and also due to a general dependency on a certain quality and volume of data that's only actually available in our staging environment. Then also, right, we've got a CI-CD system. As I showed, we're using GitLab here, but imagine a system that can go run your scans, run your security checks, run whatever lint things you need to do, and then also run all of your unit tests whenever a developer makes a change set into your develop branch. And then at Demandbase, we're doing a weekly release branch where, say, every Monday, we cut a release candidate from develop, deploy that up to staging, spend Tuesday and Wednesday running a suite of integration, regression, load tests, manual validation, all that, and then deploy out every Thursday to production. All right, so with that in mind, let's start to shift left. So our first goal was we wanted to allow UX, say, designers on the team or PMs or even other developers on the team to be able to really fully review a branch change before merging it in. But of course, right, our designers and our PMs often don't know how to use Git. They don't want to learn how to use it. Even for a developer, we don't want you to have to stash your changes, check out your peers branch, load that locally and all that just to see what all is going on. So let's start to shift left. And here's an example of the GitLab pipeline that we ended up building up. So imagine I've got my feature branch, I'm making an MR to develop. Like I said, we're going to go and run all of these checks and scans and unit tests and all that. But then importantly, at the end of this process, we're going to go build up the full application and push it off to our staging origin, but in this specific path, right, off into this branches and then a named path. And then in order to support this from the CDN side of things, we introduced Lambda at Edge, which allows CloudFront to run some serverless function either in the request coming in or the response going out. And this allowed us to do some simple branch rewrite. So if I'm trying to go and access a particular branch deploy CloudFront, the Lambda at Edge will catch this, will rewrite the path and it'll go load the proper origin. And then finally, right, this wires up really nicely with GitLab. So our designers, our PMs can just go look at an MR, click the view app button and they're off to the races. All right. So great progress, but we want to keep shifting left. And like I mentioned, the name of the talk is Canaries in the CloudFront. So let's talk canaries now. So we wanted to support canary releases of our latest develop into the staging environment so that we could run this full integration, regression suite, load suite, all those test suites ahead of ever even cutting a release candidate. So again, right, our GitLab pipeline can support this. So now after my changes merge to develop, again, we'll run scans, we'll run tests, we'll build libraries, all that. We'll go auto deploy the build out to the dev environment, but then also we're going to go and build up this canary package and we're going to push it off to the staging environment again in some canary path. And doing this then required us to be a little more complex with the way we're handling this Lambda at Edge in front of our CloudFront. Namely, because we wanted the ability to move automation users back and forth from canary to the current build, we introduced a cookie schema and introduced this into the edge cache key such that a user requesting a web.demandbase.com will either actually load the canary build or the current build based on the presence of these cookies. And then in our React app, we wanted to add a little flexibility around this. So we're using launch darkly pretty extensively for feature flag management at demand base. We continue to do that here, but then we also wanted to add some method where a developer could easily flip back and forth from the current build to the canary build. And so the React bootstrap ends up doing some checks against the feature flags, against cookie state and determining does it need to go and reload from CloudFront, which is going to enable the edge cache to check for those cookies and load the proper build. All right. So we've got canaries now all the way up to staging. We're using them to test the build ahead of ever even cutting a release branch. So now which way do we go? Which way is left from here if we want to keep shifting left? Well, number one, right, we originally thought, let's go and push this canary deploy out to production. So similar to what we do for a lot of our backend services at demand base, this will enable us to go, say, deploy the green build out, run some subset of traffic against it, understand is it performing correctly, and then cut all traffic over. However, this is where I think it's important to think about, you know, who are your customers and what is the market that you're in? Demand base is in B2B SaaS, right? And so in particular, this becomes a little more complicated because we always want to avoid multiple users within the same customer tenant experiencing different versions of the application. And like I said, right, we use feature flagging heavily, but for the most part, this is always tenant specific, right? So a particular customer might get a alpha feature turned on, but it's quite rare to go and turn things on for individual users and have them have a different experience than other users in that same customer tenant. And then finally, we do have this large internal user pool, demand based employees who are both using our platform for demand base's own marketing and sales and advertising, but also demand based employees that are supporting our customers. And as we kind of talk to the PM team, we talked to some of the implementation and customer support teams, we realized that these workflows are really mission critical. And so it's not a good candidate pool to say, hey, we're going to push you off into a new build and then we'll cut everybody over once we're happy. So then we thought about, well, where should we go from here? You know, what does shift left mean in this case? So for us, this meant targeted integration tests against our branched employees. Like I mentioned, right, we have this full rich integration, load regression, all these test suites, and they're slow and they're somewhat expensive and they require stable data and a stable backend to be running against. But so now with our initial shift left, we're going to go run all those suites nightly against latest develop, but already, right, it's too late. If there's a problem, if there's a bug, well, the code has already been committed. What we would really like is to find those issues on the merge request itself before it ever even hits the develop branch. Also, we noticed, well, individual changes are often targeted to some particular functional area, which would enable us to go and say, let's run some subset of the integration test, right? Let's not take the full three hours, but maybe 15, 20, 30 minutes worth of some subset of the suite might really help here. So Lambda at Edge enables us to go and direct our automation suites on demand, right, to the canary builder, to the current build. So it was not much work then to say, let's also make this support direct into a branch build in order to run some subset of the tests. So our last step here that we're actively working on is wiring up our CI CD system that'll allow developers to pick and choose what parts of the integration test suite they want to run against the branch MRs before they ever even submit them for peer review. So that's all I've got for today. Thank you so much for taking the time and joining. If you take a look at the link here, Demandbase is pretty much always hiring frontend and backend in the US and in India. So if any of this stuff interests you or you want to reach out more, please take a look at the link or reach out to me. Thank you so much and have a great day.
