Canaries in the CloudFront

Rate this content
Bookmark
Slides

* intro to DB application architecture (gateway + backend, multi-module react app + component library, etc)
* Deep dive into CI/CD pipeline to do component library build + publish, app deploys pre canary (branch, dev, staging, prod), Lambda@Edge role
* Side  note - multi module applications, enabling the right level of modularity, and B2B SaaS vs B2C
* First canary problem - run full FE regression against stable (release candidate) backend early and often - before FE release candidates are cut
* Second canary problem - run pointed FE regression even earlier - before MRs are even merged
* Third canary question - what does this look like in production? Once again, B2B SaaS vs B2C
* End result - Canaries in the CloudFront! 

Vijay Ramesh
Vijay Ramesh
12 min
06 Jun, 2023

Comments

Sign in or register to post your comment.

Video Summary and Transcription

Today's Talk covers designing frontend CI/CD systems, integrating CDNs, and the impact of business type on frontend teams. The speaker shares their experience with CDN integration and its impact on their CI/CD strategy. They discuss canary releases and the challenges of pushing canary deploys to production. Additionally, they highlight the importance of integration testing for shift left and detecting issues before merging to the develop branch.

Available in Español: Canarios en el CloudFront

1. Introduction to Canaries in the Cloud Front

Short description:

Today, I will talk about designing frontend CI CD systems for faster and less risky development. I will also discuss the importance of considering your content delivery network in your application architecture. Lastly, I will explore how your business type and target market can impact your frontend teams' shift left approach.

Hi, everybody. Thanks for joining. My name's Vijay Ramesh. I'm a VP of application development at Demandbase. And today I wanted to talk to you a little bit about something I like to call canaries in the cloud front.

Before we jump in, we only got about 10 minutes today and that's not a lot of time. But in a way it is. There's a lot of content out there. There's a lot of things to learn, a lot of things to dig into and not that much time in any given day. So let me cut to the chase first and talk a little bit about what am I going to learn if you spend the next nine and a half minutes here listening to my talk.

So number one, I want to talk about what it means to design your frontend CI CD systems in a way that enable your teams to move faster with less risk. And as a part of this, what does it mean to think about your content delivery network as an integral part of your application architecture? And then finally, I want to touch a bit on how the type of business that you run, the type of product you make, the type of customers and market that you serve, how that can help determine what shift left could mean for your frontend teams.

2. Introduction to CDN Integration and CI/CD Strategy

Short description:

I'm a VP of application development at Demandbase with over 20 years of experience. We introduced a CDN to our architecture, leveraging CloudFront for edge caching. This impacted our CI/CD strategy as we needed to invalidate the cache with each new build. Our CI/CD system includes dev and staging environments, Selenium integration tests, and GitLab for scans, security checks, and unit tests. We follow a weekly release branch process, deploying to staging for testing before going to production. To enable easier branch review, we built a GitLab pipeline that runs checks, scans, and tests, and pushes the application to the staging origin. We also introduced Lambda at Edge for branch rewriting. This integration with GitLab allows designers and PMs to easily review branch changes.

And just briefly, like I said, so I'm a VP of application development at Demandbase. Over 20 years now, full stack web app type experience, big focus on building rich applications atop of big data and ML platforms, and today at Demandbase I lead three of our application engineering groups.

So before we start, let's actually go maybe one step from before we start, and an architecture diagram like this might be familiar to people who have been doing this for a long time, right? So think in the pre-CDN days, maybe even in the pre-single-page app days, it would be common to have some monolithic system like a Rails app, that app is going to go and be fronted by some sort of load balancer, and requests from a web client are going to hit the load balancer and go back to the backend, and the backend is going to serve up its own frontend. So let's go maybe one step forward here and introduce a CDN, right? So at Demandbase we're using AWS, so we use CloudFront here, and what do we get when we go and introduce a CDN like this? The biggest bang for the buck I think is this idea of edge caching, so if Demandbase is building up its React app and storing it off in US East 1, say in Virginia, meanwhile me as a user coming from Berkeley, California, I want to be able to fetch those static assets from an edge cache that is close to my location, and CloudFront enables this pretty natively, so as a user coming in from California, I'm going to go hit the edge cache say in US West Maybe we've already got the latest build there so I don't even need to fall on back down to the origin, or in the case I need to, the origin is going to go and warm up the cache on the edge so the next user, say coming from San Francisco, gets the full application. And already here, you need to start thinking about, well how does introducing this CDN technology impact your CI, CD strategy. In this case, right, you need to make sure that you're going to invalidate the cache as you deploy new builds, such that a user coming, say from one of these West Coast edges, is able to pick up the latest build. And then also, let's talk a little bit about what are the pre-reqs for the CI-CD system that we started with, when we really wanted to shift left. So number one, dev and staging, right, full pre-production environments, and a way locally for a developer to go run the full frontend application but talking against, say, that staging or that dev backend. Number two, we have a large suite of Selenium integration tests, and importantly these only really will run in the staging environment, due to a general dependency on a stable backend. In the dev environment, backend teams might be testing out feature branches and things like that, and also due to a general dependency on a certain quality and volume of data that's only actually available in our staged environment. Then also, right, we've got a CI-CD system. As I showed, we're using GitLab here. But imagine a system that can go run your scans, run your security checks, run whatever lint things you need to do, and then also run all of your unit tests whenever a developer makes a change set into your develop branch. And then at Demandbase, we're doing a weekly release branch, where, say, every Monday we cut a release candidate from develop, deploy that up to staging, spend Tuesday and Wednesday running a suite of integration, regression, load tests, manual validation, all that, and then deploy out every Thursday to production.

So with that in mind, let's start to shift left. So our first goal was, we wanted to allow UX, say, designers on the team or PMs or even other developers on the team, to be able to really fully review a branch change before merging it in. But of course, right, our designers and our PMs often don't know how to use Git. They don't want to learn how to use it. Even for a developer, we don't want you to have to stash your changes, check out your peers branch, load that locally and all that just to see what all is going on. So let's start to shift left. And here's an example of the GitLab pipeline that we ended up building up. So imagine I've got my feature branch, I'm making it an MR to develop. Like I said, we're going to go and run all of these checks and scans and unit tests and all that. But then importantly, at the end of this process, we're going to go build up the full application and push it off to our staging origin, but in this specific path, right off into those branches and then a named path. And then in order to support this from the CDN side of things, we introduced Lambda at Edge, which allows CloudFront to run some serverless function, either in the request coming in or the response going out. And this allowed us to do some simple branch rewrite. So if I'm trying to go and access a particular branch deploy CloudFront, the Lambda at Edge will catch this, will rewrite the path, and it'll go load the proper origin. And then finally, right, this wires up really nicely with GitLab. So our designers, our PMs can just go look at an MR, click the view app button, and they're off to the races.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Levelling up Monorepos with npm Workspaces
DevOps.js Conf 2022DevOps.js Conf 2022
33 min
Levelling up Monorepos with npm Workspaces
Top Content
Learn more about how to leverage the default features of npm workspaces to help you manage your monorepo project while also checking out some of the new npm cli features.
Full Stack Components
Remix Conf Europe 2022Remix Conf Europe 2022
37 min
Full Stack Components
Top Content
Remix is a web framework that gives you the simple mental model of a Multi-Page App (MPA) but the power and capabilities of a Single-Page App (SPA). One of the big challenges of SPAs is network management resulting in a great deal of indirection and buggy code. This is especially noticeable in application state which Remix completely eliminates, but it's also an issue in individual components that communicate with a single-purpose backend endpoint (like a combobox search for example).
In this talk, Kent will demonstrate how Remix enables you to build complex UI components that are connected to a backend in the simplest and most powerful way you've ever seen. Leaving you time to chill with your family or whatever else you do for fun.
RedwoodJS: The Full-Stack React App Framework of Your Dreams
React Summit Remote Edition 2021React Summit Remote Edition 2021
43 min
RedwoodJS: The Full-Stack React App Framework of Your Dreams
Top Content
Tired of rebuilding your React-based web framework from scratch for every new project? You're in luck! RedwoodJS is a full-stack web application framework (think Rails but for JS/TS devs) based on React, Apollo GraphQL, and Prisma 2. We do the heavy integration work so you don't have to. We also beautifully integrate Jest and Storybook, and offer built-in solutions for declarative data fetching, authentication, pre-rendering, logging, a11y, and tons more. Deploy to Netlify, Vercel, or go oldschool on AWS or bare metal. In this talk you'll learn about the RedwoodJS architecture, see core features in action, and walk away with a sense of wonder and awe in your heart.
Automating All the Code & Testing Things with GitHub Actions
React Advanced Conference 2021React Advanced Conference 2021
19 min
Automating All the Code & Testing Things with GitHub Actions
Top Content
Code tasks like linting and testing are critical pieces of a developer’s workflow that help keep us sane like preventing syntax or style issues and hardening our core business logic. We’ll talk about how we can use GitHub Actions to automate these tasks and help keep our projects running smoothly.
Fine-tuning DevOps for People over Perfection
DevOps.js Conf 2022DevOps.js Conf 2022
33 min
Fine-tuning DevOps for People over Perfection
Top Content
Demand for DevOps has increased in recent years as more organizations adopt cloud native technologies. Complexity has also increased and a "zero to hero" mentality leaves many people chasing perfection and FOMO. This session focusses instead on why maybe we shouldn't adopt a technology practice and how sometimes teams can achieve the same results prioritizing people over ops automation & controls. Let's look at amounts of and fine-tuning everything as code, pull requests, DevSecOps, Monitoring and more to prioritize developer well-being over optimization perfection. It can be a valid decision to deploy less and sleep better. And finally we'll examine how manual practice and discipline can be the key to superb products and experiences.
Why is CI so Damn Slow?
DevOps.js Conf 2022DevOps.js Conf 2022
27 min
Why is CI so Damn Slow?
We've all asked ourselves this while waiting an eternity for our CI job to finish. Slow CI not only wrecks developer productivity breaking our focus, it costs money in cloud computing fees, and wastes enormous amounts of electricity. Let’s take a dive into why this is the case and how we can solve it with better, faster tools.

Workshops on related topic

Developing Dynamic Blogs with SvelteKit & Storyblok: A Hands-on Workshop
JSNation 2023JSNation 2023
174 min
Developing Dynamic Blogs with SvelteKit & Storyblok: A Hands-on Workshop
Top Content
Featured WorkshopFree
Alba Silvente Fuentes
Roberto Butti
2 authors
This SvelteKit workshop explores the integration of 3rd party services, such as Storyblok, in a SvelteKit project. Participants will learn how to create a SvelteKit project, leverage Svelte components, and connect to external APIs. The workshop covers important concepts including SSR, CSR, static site generation, and deploying the application using adapters. By the end of the workshop, attendees will have a solid understanding of building SvelteKit applications with API integrations and be prepared for deployment.
Deploying React Native Apps in the Cloud
React Summit 2023React Summit 2023
88 min
Deploying React Native Apps in the Cloud
WorkshopFree
Cecelia Martinez
Cecelia Martinez
Deploying React Native apps manually on a local machine can be complex. The differences between Android and iOS require developers to use specific tools and processes for each platform, including hardware requirements for iOS. Manual deployments also make it difficult to manage signing credentials, environment configurations, track releases, and to collaborate as a team.
Appflow is the cloud mobile DevOps platform built by Ionic. Using a service like Appflow to build React Native apps not only provides access to powerful computing resources, it can simplify the deployment process by providing a centralized environment for managing and distributing your app to multiple platforms. This can save time and resources, enable collaboration, as well as improve the overall reliability and scalability of an app.
In this workshop, you’ll deploy a React Native application for delivery to Android and iOS test devices using Appflow. You’ll also learn the steps for publishing to Google Play and Apple App Stores. No previous experience with deploying native applications is required, and you’ll come away with a deeper understanding of the mobile deployment process and best practices for how to use a cloud mobile DevOps platform to ship quickly at scale.
Full Stack GraphQL In The Cloud With Neo4j Aura, Next.js, & Vercel
GraphQL Galaxy 2021GraphQL Galaxy 2021
161 min
Full Stack GraphQL In The Cloud With Neo4j Aura, Next.js, & Vercel
WorkshopFree
William Lyon
William Lyon
In this workshop we will build and deploy a full stack GraphQL application using Next.js, Neo4j, and Vercel. Using a knowledge graph of news articles we will first build a GraphQL API using Next.js API routes and the Neo4j GraphQL Library. Next, we focus on the front-end, exploring how to use GraphQL for data fetching with a Next.js application. Lastly, we explore how to add personalization and content recommendation in our GraphQL API to serve relevant articles to our users, then deploy our application to the cloud using Vercel and Neo4j Aura.

Table of contents:
- Next.js overview and getting started with Next.js
- API Routes with Next.js & building a GraphQL API
- Using the Neo4j GraphQL Library
- Working with Apollo Client and GraphQL data fetching in Next.js
- Deploying with Vercel and Neo4j Aura
MERN Stack Application Deployment in Kubernetes
DevOps.js Conf 2022DevOps.js Conf 2022
152 min
MERN Stack Application Deployment in Kubernetes
Workshop
Joel Lord
Joel Lord
Deploying and managing JavaScript applications in Kubernetes can get tricky. Especially when a database also has to be part of the deployment. MongoDB Atlas has made developers' lives much easier, however, how do you take a SaaS product and integrate it with your existing Kubernetes cluster? This is where the MongoDB Atlas Operator comes into play. In this workshop, the attendees will learn about how to create a MERN (MongoDB, Express, React, Node.js) application locally, and how to deploy everything into a Kubernetes cluster with the Atlas Operator.
Azure Static Web Apps (SWA) with Azure DevOps
DevOps.js Conf 2022DevOps.js Conf 2022
13 min
Azure Static Web Apps (SWA) with Azure DevOps
WorkshopFree
Juarez Barbosa Junior
Juarez Barbosa Junior
Azure Static Web Apps were launched earlier in 2021, and out of the box, they could integrate your existing repository and deploy your Static Web App from Azure DevOps. This workshop demonstrates how to publish an Azure Static Web App with Azure DevOps.