How to Build CI/CD Pipelines for a Microservices Application

Rate this content
Bookmark

Microservices present many advantages for running modern software, but they also bring new challenges for both Deployment and Operational tasks. This session will discuss advantages and challenges of microservices and review the best practices of developing a microservice-based architecture.

We will discuss how container orchestration using Kubernetes or Red Hat OpenShift can help us and bring it all together with an example of Continuous Integration and Continuous Delivery (CI/CD) pipelines on top of OpenShift.

Sasha Rosenbaum
Sasha Rosenbaum
33 min
01 Jul, 2021

Comments

Sign in or register to post your comment.

Video Summary and Transcription

This Talk discusses the benefits of microservices and containers for building CI-CD pipelines. It explains how container technology enables portability and scalability. The challenges of microservices include network communication and testing in isolation. The Talk introduces Tacton, a cloud-native CICD pipeline for Kubernetes, and highlights the use of GitOps and Argo CD. It also discusses the importance of maintaining referential integrity between microservices and the evolving role of operators in the DevOps world.

1. Introduction and Background

Short description:

Hi, I'm Sasha Rosenbaum, and today I'm presenting on building CI-CD for microservices applications. I have been in development, operations, dev route, and customer success. I've been involved in DevOps days since 2014. You can find me on Twitter as devineops.

Hi, I'm Sasha Rosenbaum, and today I'm presenting to you on building CI-CD for microservices applications. So first, just a little bit of an introduction about me. I am currently a Team Lead on a managed OpenShift BlackBell team at Red Hat. And in my career, I have been in development, in operations, briefly in dev route, and I spent quite a bit of time in customer success. I still consider myself a developer. But as you can see, I've done a lot of different things along the 15 years that I've been in this industry.

I've been involved in DevOps days since 2014. You can see on the chart as well. And you can find me on Twitter as devineops. I post a lot of cat pictures and some Twitter, some technical hot takes. So please feel free to follow me on Twitter.

2. Microservices and Their Benefits

Short description:

So let's talk about microservices. Once upon a time, there was a monolith, and it was easier to deal with because you had full control. However, there were challenges, such as collaboration between different teams and the need for regression testing. The solution to these challenges became microservices, which involve modularizing different parts of the application and breaking them into separate services. Each microservice is ideally supposed to own its own data. Microservices separate by business logic.

So let's talk about microservices. As I was building this presentation, I was thinking that in addition to how to do things, it's also important to talk about why we're doing things. And especially if we are talking from a developer standpoint, it's important to talk about why we even use microservices, why we use containers, and dive into kind of overview of the entire architecture.

So let's start with this. Once upon a time, there was a monolith, right? And the monolith was pretty, and we actually enjoyed working with a monolith, and everything was relatively okay. And to tell you a little secret that no one likes to talk about anymore, is that monolith was actually easier to deal with, because monolith, you had full control over all the parts of your application, all of your dependencies, and all of your things. And also because distributed systems are hard, and that's just a universal truth.

But monolith also had challenges and these challenges were among many. So many different teams needed to collaborate. So you had to collaborate across different parts of a company. And if the company was large, that often meant that something like a large airline.com became a behemoth, became a huge beast and all the different teams had to be involved in building that huge application. Everyone had to be using the same technology. So if one team really liked Java and another team really likes C-sharp, they had to choose between them. They had to use both, or same with different OS types and things like that. Every change required a regression testing of the entire system, which can be quite time-consuming. Deployment therefore was slow, right? Because teams were often running into merge hell and scaling was difficult. So scaling a huge beast of an application is usually not a picnic. So because we had challenges where there's challenges, there's always solutions. And in this particular case the solution became microservices.

So what's really microservices? We started with an application, and then we started talking about proper patterns of development. So we modularized different parts of application. And then we started talking about breaking these different modules into different services. And as we went, it became a network of services. In proper implementation, each microservice also is supposed to own their own data. In practice, this doesn't always happen, but this is the implementation best practice for each microservice to also own the database. So I often hear questions about service oriented architecture versus microservices. So if you're on the left of this diagram, and you have a data layer, and you have a business logic layer, and you have a web layer, even if these three layers are separate, you're in microservices. Separating by vertical concerns is still not microservices. Microservices separate by business logic. So in this case, for instance, authentication would be a separate microservice.

QnA

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Levelling up Monorepos with npm Workspaces
DevOps.js Conf 2022DevOps.js Conf 2022
33 min
Levelling up Monorepos with npm Workspaces
Top Content
Learn more about how to leverage the default features of npm workspaces to help you manage your monorepo project while also checking out some of the new npm cli features.
Automating All the Code & Testing Things with GitHub Actions
React Advanced Conference 2021React Advanced Conference 2021
19 min
Automating All the Code & Testing Things with GitHub Actions
Top Content
Code tasks like linting and testing are critical pieces of a developer’s workflow that help keep us sane like preventing syntax or style issues and hardening our core business logic. We’ll talk about how we can use GitHub Actions to automate these tasks and help keep our projects running smoothly.
Fine-tuning DevOps for People over Perfection
DevOps.js Conf 2022DevOps.js Conf 2022
33 min
Fine-tuning DevOps for People over Perfection
Top Content
Demand for DevOps has increased in recent years as more organizations adopt cloud native technologies. Complexity has also increased and a "zero to hero" mentality leaves many people chasing perfection and FOMO. This session focusses instead on why maybe we shouldn't adopt a technology practice and how sometimes teams can achieve the same results prioritizing people over ops automation & controls. Let's look at amounts of and fine-tuning everything as code, pull requests, DevSecOps, Monitoring and more to prioritize developer well-being over optimization perfection. It can be a valid decision to deploy less and sleep better. And finally we'll examine how manual practice and discipline can be the key to superb products and experiences.
Why is CI so Damn Slow?
DevOps.js Conf 2022DevOps.js Conf 2022
27 min
Why is CI so Damn Slow?
We've all asked ourselves this while waiting an eternity for our CI job to finish. Slow CI not only wrecks developer productivity breaking our focus, it costs money in cloud computing fees, and wastes enormous amounts of electricity. Let’s take a dive into why this is the case and how we can solve it with better, faster tools.
The Zen of Yarn
DevOps.js Conf 2022DevOps.js Conf 2022
31 min
The Zen of Yarn
In the past years Yarn took a spot as one of the most common tools used to develop JavaScript projects, in no small part thanks to an opinionated set of guiding principles. But what are they? How do they apply to Yarn in practice? And just as important: how do they benefit you and your projects?
In this talk we won't dive into benchmarks or feature sets: instead, you'll learn how we approach Yarn’s development, how we explore new paths, how we keep our codebase healthy, and generally why we think Yarn will remain firmly set in our ecosystem for the years to come.
End the Pain: Rethinking CI for Large Monorepos
DevOps.js Conf 2024DevOps.js Conf 2024
25 min
End the Pain: Rethinking CI for Large Monorepos
Scaling large codebases, especially monorepos, can be a nightmare on Continuous Integration (CI) systems. The current landscape of CI tools leans towards being machine-oriented, low-level, and demanding in terms of maintenance. What's worse, they're often disassociated from the developer's actual needs and workflow.Why is CI a stumbling block? Because current CI systems are jacks-of-all-trades, with no specific understanding of your codebase. They can't take advantage of the context they operate in to offer optimizations.In this talk, we'll explore the future of CI, designed specifically for large codebases and monorepos. Imagine a CI system that understands the structure of your workspace, dynamically parallelizes tasks across machines using historical data, and does all of this with a minimal, high-level configuration. Let's rethink CI, making it smarter, more efficient, and aligned with developer needs.

Workshops on related topic

Decomposing Monolith NestJS API into GRPC Microservices
Node Congress 2023Node Congress 2023
119 min
Decomposing Monolith NestJS API into GRPC Microservices
Workshop
Alex Korzhikov
Alex Korzhikov
The workshop focuses on concepts, algorithms, and practices to decompose a monolithic application into GRPC microservices. It overviews architecture principles, design patterns, and technologies used to build microservices. It covers the theory of the GRPC framework and protocol buffers mechanism, as well as techniques and specifics of building isolated TypeScript services in the Node.js stack. The workshop includes a live use case demo of decomposing an API application into a set of microservices. It fits the best architects, tech leads, and developers who want to learn microservices patterns.
Level: AdvancedPatterns: DDD, MicroservicesTechnologies: GRPC, Protocol Buffers, Node.js, TypeScript, NestJS, Express.js, PostgreSQL, TurborepoExample structure: monorepo configuration, packages configuration, common utilities, demo servicePractical exercise: refactor monolith app
Deploying React Native Apps in the Cloud
React Summit 2023React Summit 2023
88 min
Deploying React Native Apps in the Cloud
WorkshopFree
Cecelia Martinez
Cecelia Martinez
Deploying React Native apps manually on a local machine can be complex. The differences between Android and iOS require developers to use specific tools and processes for each platform, including hardware requirements for iOS. Manual deployments also make it difficult to manage signing credentials, environment configurations, track releases, and to collaborate as a team.
Appflow is the cloud mobile DevOps platform built by Ionic. Using a service like Appflow to build React Native apps not only provides access to powerful computing resources, it can simplify the deployment process by providing a centralized environment for managing and distributing your app to multiple platforms. This can save time and resources, enable collaboration, as well as improve the overall reliability and scalability of an app.
In this workshop, you’ll deploy a React Native application for delivery to Android and iOS test devices using Appflow. You’ll also learn the steps for publishing to Google Play and Apple App Stores. No previous experience with deploying native applications is required, and you’ll come away with a deeper understanding of the mobile deployment process and best practices for how to use a cloud mobile DevOps platform to ship quickly at scale.
Decoupling in Practice
Node Congress 2023Node Congress 2023
102 min
Decoupling in Practice
WorkshopFree
Chad Carlson
Chad Carlson
Deploying decoupled and microservice applications isn't just a problem to be solved on migration day. Moving forward with these architectures depends completely on what your team's workflow experience will look like day-to-day post-migration.
The hardest part of this can often be the number of vendors involved. Some targets are best suited for specific frontend frameworks, while others are more so for CMSs and custom APIs. Unfortunately their assumptions, workflows, APIs, and notions of security can be quite different. While there are certain advantages to relying on a strict contract between apps – where backend and frontend teams work is limited to a single vendor – this isn't always realistic. This could be because you're still experimenting, or simply the size of your organization doesn't allow for this kind of specialization just yet.
In this workshop, you'll have a chance to explore a different, single vendor approach to microservices using Strapi and Next.js as an example. You'll deploy each app individually, establishing a workflow from the start that simplifies customization, introducing new features, investigating performance issues, and even framework interchangeability from the start.
Structure:- Getting started- Overview of Strapi- Overview of Platform.sh workflow- Deploying the project- Switching services- Adding the frontend
Prerequisites:- A Platform.sh trial account created- The Platform.sh CLI installed
MERN Stack Application Deployment in Kubernetes
DevOps.js Conf 2022DevOps.js Conf 2022
152 min
MERN Stack Application Deployment in Kubernetes
Workshop
Joel Lord
Joel Lord
Deploying and managing JavaScript applications in Kubernetes can get tricky. Especially when a database also has to be part of the deployment. MongoDB Atlas has made developers' lives much easier, however, how do you take a SaaS product and integrate it with your existing Kubernetes cluster? This is where the MongoDB Atlas Operator comes into play. In this workshop, the attendees will learn about how to create a MERN (MongoDB, Express, React, Node.js) application locally, and how to deploy everything into a Kubernetes cluster with the Atlas Operator.
Azure Static Web Apps (SWA) with Azure DevOps
DevOps.js Conf 2022DevOps.js Conf 2022
13 min
Azure Static Web Apps (SWA) with Azure DevOps
WorkshopFree
Juarez Barbosa Junior
Juarez Barbosa Junior
Azure Static Web Apps were launched earlier in 2021, and out of the box, they could integrate your existing repository and deploy your Static Web App from Azure DevOps. This workshop demonstrates how to publish an Azure Static Web App with Azure DevOps.