Building and Operating a Modern Composable Monolith

Rate this content

It all started with the monolith’s fission into microservices. This set logical and physical boundaries, fusing the infrastructure and software dimensions. While microservices simplified how teams develop independently, it added extra complexities around performance, correctness, general management, and ultimately a slower development cycle. In this talk, we will dive deep into how to architect and implement a Fastify-based composable monolith, cross-service network-less communication, and how to handle efficient development across teams while deploying a single artifact in Kubernetes. Does it sound like magic?Let's discover the trick together!

19 min
15 Feb, 2024

Video Summary and Transcription

In this talk, Luca introduces the concept of the modern composable monolith and discusses the challenges of microservices. He emphasizes the importance of developing modular monoliths and introduces Fastify and the Platformatic Runtime as tools for this approach. The Platformatic Runtime simplifies running modular, monolithic applications, and standardizing interfaces and communication is crucial in this context. Lastly, Luca introduces Meraki as a UI-driven tool for building composable monoliths and invites developers to join their marketplace for revolutionizing the operational experience of microservices.

Available in Español

1. Introduction to the Modern Composable Monolith

Short description:

Hello everyone, I'm Luca, co-founder and CEO of Platformatic. Today we have microservices, but where is the future taking us? In this talk, I will try to trigger your curiosity into a new concept and to sneak peek into some of the material that at Platformatic we already built on this topic. Microservices bridge the logical boundaries with the physical boundaries, bringing a technology, cultural, and people transformation. They bring an insane performance boost and more fault tolerance, but hurt the performance and make correctness harder.

Hello everyone, I'm extremely excited to be here at JS Nation today and to take you on a journey, the journey of the modern composable monolith. Let me start introducing myself, I'm Luca, co-founder and CEO of Platformatic, and together with my friend Matteo Collina and our amazing team, we are on a mission to completely change the world of building Node.js application in modern times.

You can connect with me and please do it on any social media and you can find me under Luca Maraschi, I'm excited to connect with you. But let me start with our journey. My grandmother Piera always told me life is like a spinning wheel and at the time, trust me, I didn't get it. But today I want to show you and prove you that she was right. Let's start.

Like every story that we heard in the past years, especially in Node, is that once upon a time there was the monolith and now microservices. Well, today, let's start from where we left the story. Today we have microservices, but where is the future taking us? In this talk, I will try to trigger your curiosity into a new concept, but especially to sneak peek into some of the material that at Platformatic we already built on this topic and to connect directly to ask any question.

But let's start with where the story ended and was left about microservices. So microservices, they bridge the logical boundaries with the physical boundaries. And what does that mean is that they basically are not only a technology transformation, but also a cultural transformation and a people transformation, like we say in any organization. Microsoft has always, when we think about it, we think about all these hexagons, right? Being composed with each other and then being streamlined into APIs. Because we are in the time of modern APIs and of the API economy, like my friends at Kong always say. And I agree with them.

We are in the midst of the API economy. And on one side, we have been extremely capable and driven by changing the world of APIs. Still, we got a little bit stagnant in how we build the logic that supports all these APIs. The core of everyone's business. And so microservices on one side brought an insane performance boost, right? Because we can scale independently, we think about micro particles of the business logic in a much more isolated way, right? They became more fault tolerance because we don't have these huge transactions that are unified into a single set of business logic, but they are distributed across multiple nodes. And if one node goes down, we can actually restore it because we were told they need to be stateless. So we started creating these abstraction boundaries. We create this facade that allow us to swap in and swap out the different implementations of the service without disrupting clients, right? And this sounds extremely exciting and positive. However, in every good story there is an however, you know, right? This actually hurt the performance because now you have a lot of services that imagine in Kubernetes or in any other system, they are basically distributed across multiple nodes and these nodes needs to communicate. And how they communicate is called the network, right? On the other side, correctness got way harder. It's a distributed system. It's very hard to manage consistent state across all these different services. And especially when I say state, I mean, logical state and physical state. Is everything equally distributed? Is everything developing the same way? And just pin this concept for later.

2. Challenges of Microservices and the Eureka Moment

Short description:

They become extremely hard to manage because they are like a distributed system. It impacts directly developers, making the development lifecycle slower and adding frustrations. We couple our application logic to the network layout, making it hard to untangle. The solution is to look backward at the composable monolith, a concept not new but endorsed by Google in a white paper.

They become extremely hard to manage because we just said it, right? They are like a distributed system. They are all independently developed, independently deployed, independently managed, right? And so, this actually causes a huge freeze in the APIs, right? Because we just said before, the APIs, they are this aggregation of all these microservices. And so, at the end of the day, what happened is that it impacts directly developers. It makes the development lifecycle way slower and it adds frustrations to developers because there are more barriers to overcome before we can get our microservices to production.

But I would say even more, we couple our application logic, our applications to the network layout with a network taxonomy, and these become extremely hard to untangle. And also, it makes versioning extremely complicated because application binaries are released independently. Every single microservice has its own lifecycle. Have you ever wondered which version should this service have? That's actually a very good question. But there was a eureka moment. There was a time that we said, hmm, we saw the light. And that was the time of the composable monolith.

So, what is the solution to all these problems? Well, it's probably to look backward and look where we started, right? We all started the bashing and going that basically the monolith represented Java and .NET, right? And at the end, if you all remember, those technologies were deployed like a tarball that was SFTPs on an application server. And it was working, right? But what actually was not fitting was the development model, how this bundle was actually created. And so, this is not a new concept, right? And I'm not the first one. We are not, like, crazy saying about this composable monolith. But Google actually wrote a white paper saying that the future of modern development in the cloud is with a composable monolith. This is the white paper. I highly recommend you to download it, read it. It's a phenomenal paper. And I personally think it gives a phenomenal view on modern ways of interpreting software development.

3. Key Points in Developing Modular Monoliths

Short description:

The first point is to write monolithic applications that are modularized into components. The second point is to leverage a runtime that can assign logical components to physical resources. Deploy applications atomically, allowing for easier versioning and consumption. This opens many doors, and there are already frameworks in the Node ecosystem.

Let me actually pick three important points of this whole story. Well, the first one is that basically to write monolithic applications that are modularized into components. So, we learn that the concept of components, right, from the front end, if you think about React, and this was phenomenal because every component has their own logic, their own scope. But they were all bundled into a single deployable. And that was the website, right?

The other point is to leverage a runtime that basically dynamically and automatically can assign logical components to physical resources, which means basically take this monolith, this composable component-based monolith, and assign resources to each component so that these components can actually be run as they were independent. But deploy applications atomically, so not independent deployments. One single monolithic deployment, one huge tarball. Because this will actually lead to a way easier way of versioning and to consume this version. I think this is actually opening many, many doors, right? And if you think Node, in Node, in the Node ecosystem, there are already frameworks.

4. Fastify and the Platformatic Runtime

Short description:

Fastify is the elegant solution to developing applications in this new way. Platformatic built a runtime, open source for the past six months, that allows independent components to be run as a single piece. It's recommended to try out Platformatic Runtime.

And I'm not actually just mentioning Fastify because I love Fastify, but I think it's elegantly representing the solution, probably the only solution, to this new way of developing applications. You can't do it with Express. You can do it, but it's ugly. And many other frameworks they tried, but Fastify elegantly was able to actually stitch all this concept together. So in Platformatic, we clearly love Fastify, and we said we needed to build a runtime that does exactly what Google was suggesting, to take all these independent components and run them if they were like a single piece. But let's say multi-try that, just to use like a provocative statement. And so we built Platformatic Runtime and it's already open source for the past six months. So please go and use it.

5. Solving Complexity with Platformatic Runtime

Short description:

Platformatic Runtime simplifies the complexity of running modular, monolithic applications. Fastify is designed around the concept of plugins and routes, allowing for a well-structured composable monolith. Platformatic takes this approach further, providing an easy-to-use solution. npm i platformatic to get started.

And what with Platformatic Runtime we're trying to solve is the complexity of running modular, monolithic applications, where every module represents a single feature deployed under a single bundle, developed independently by many people, like you can have 10 teams that they develop this thing, but at the end it goes into a single container. And Platformatic Runtime offers you the HTTP interface and the entry point to all of this, plus the management of running all these components as independent threads.

So when we welcome modular monoliths and we look at how we can do it in Node, I just mentioned like a minute ago about Fastify. Fastify was designed around this concept. Plugins, routes, you can actually bundle your plugin which contains routes and then you can take a single Fastify application that uses all these independent plugins. And the determinism on our deal, for example, is able to load deterministically all these plugins, allows you actually to create this very well organized, very well structured composable monolith.

And as I was mentioning, Fastify is our HTTP framework, you have all these plugins that can be npm packages or just single files into a single project, doesn't really matter. But this is kind of like the macroscopic view on how you can actually achieve that. So the solution is already there. In truth has to be told, in Platformatic we kind of took this approach to the next logical step, I highly recommend you to go and check Platformatic, easy-peasy to do, npm i platformatic and you're going to get the whole shebang ready to be used on your machine.

6. Standardizing Interfaces and Communication

Short description:

Modules in a modular monolith system need to communicate with each other through weakly referenced and weakly coupled interfaces. TypeScript is not the recommended solution. The RFC 9110 explains how the semantic split between HTTP 1 and 2 protocols makes the wiring loosely coupled. Co-founder Matteo discusses achieving performance between modules using networkless HTTP in a talk at NodeConf EU. The modules in our platformatic runtime communicate through HTTP without using any network.

But the major problem of a modular monolith system is standardizing the interfaces between modules, right? Because now I have all these modules, but how do they communicate with each other? Like you cannot strongly reference them or else you're going to tangle them. So they need to be weakly referenced, weakly coupled. But what kind of interface to use?

Well, many might say I use TypeScript, but that's not really the answer in my own opinion. So if you look at the RFC 9110, basically related to HTTP, there is a very interesting piece that says that the difference between the semantic split between HTTP 1 and 2 protocols is done to actually make the wiring loosely coupled and just an implementation detail. And that's actually very interesting because here I'm going to just reference my dear friend and co-founder Matteo. He had this brilliant talk and he's going to answer to the question of how do we actually achieve that insane performance between all these modules? And he spoke at NodeConf EU just last year about networkless HTTP and answers to that kind of like RFC puzzle that I just posted in the previous slide. I highly suggested to check this video. And what he's going to basically reveal is the secret behind our platformatic runtime. How do we actually make all these modules to communicate with each other? Well, these modules in reality, they are loosely coupled with each other and so they communicate through HTTP but without using any network. Check the talk.

7. Building Composable Monoliths with Meraki

Short description:

LEGO inspired us to build our stackables, enabling creativity and the ability to compose existing and new bricks. Meraki, an Electron app, provides a UI-driven experience to build composable monoliths. It allows developers to publish their own views on how things should be built and offers a wizard-driven experience for easier and more entertaining development. Meraki bundles compositions for distribution into our runtime, which can run stackables as independent components. Join our marketplace and revolutionize the operational experience of microservices with composable monoliths.

And that was a great segue to the next step in the interface and composability. Well, if we remove the implementation detail on how we make all these modules to communicate with each other, if we found the wiring, we needed to find a way to make them really composable. And when I think about composability, well, it comes to mind one of my passions. LEGO. You can see some here. I love LEGO. This is a small one. I have a lot of LEGO. One is just in the back and next to my guitar. And what is beautiful about LEGO and we were highly inspired is how these bricks are composed with each other. So people think that the LEGO is just a 4x2 brick, but actually LEGO patented the way they clip together. That is this geometrical way of connecting these four dots or two dots with the central bigger dot. Phenomenal way. We got very inspired to build our stackables, because we said building modern applications should just be like building a LEGO set. It should enable creativity while giving the structure to build new bricks, but also compose existing bricks with new bricks or existing bricks with existing bricks.

And so we build a Meraki. Meraki means in Greek creativity, and for us was extremely important to re-enable creativity, but also bring back the ease of building something that we are passionate about and we can be proud of. So Meraki is an Electron app that brings a purely UI driven experience to build composable monoliths. And here you see some interesting concepts like, for example, the platformatic service that is one of our base stackable templates, and you can put any plugin that you want on top. And many companies are already using this approach because they believe it is the future of scaling practices and development in their organization. So the way we actually interpret is that why don't we enable, like Brickset in LEGO, why don't we actually create an environment where people can actually publish their own view on how things should be built? They give an easy bootstrap to consumers, to other developers to build applications using their own best practices.

And the experience that we imagined was purely wizard driven because we believe that just using, you know, our clicking experience could make way easier and more entertaining and more actually fun to build this composable monoliths. Having also solving some of the complexities, like, for example, the eternal problem of building configurations or having some defaults. And at the end of the day, what we imagine is that Meraki was able to create that composition, that building experience, and then to bundle them for distribution into our runtime. Because as you remember, our runtime is able to take the stackables, these platformatic applications and run them as they were independent to components. You can find Meraki on our GitHub. It's open source. If you want to add a new feature, please send us a PR. And with no further wording, I just invite you to download it as is available now and use it and join our marketplace and join this revolution of the composable monolith that we believe is going to change the operational experience of microservices, especially Node.js. I invite you to explore Platformatic and clearly Fastify and especially to connect with us. And I would be more than happy to answer to any question you have and to explain even deeper why our approach and composable monolith approach is the future of a leaner developer experience and operational experience in Node.js ecosystem, especially in enterprise. Ciao!

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

DevOps.js Conf 2021DevOps.js Conf 2021
33 min
How to Build CI/CD Pipelines for a Microservices Application
Top Content
Microservices present many advantages for running modern software, but they also bring new challenges for both Deployment and Operational tasks. This session will discuss advantages and challenges of microservices and review the best practices of developing a microservice-based architecture.We will discuss how container orchestration using Kubernetes or Red Hat OpenShift can help us and bring it all together with an example of Continuous Integration and Continuous Delivery (CI/CD) pipelines on top of OpenShift.
JSNation Live 2021JSNation Live 2021
21 min
Micro-scopes – How to Build a Modular Modern App in a Bundled World
In this talk we will explore the great benefits of breaking a big modern app to meaningful, independent pieces – each can be built, deployed and loaded separately. We will discuss best practices and gotchas when trying to apply this microservice-like pattern to the chaotic world of the browser, and we'll see how building the right pieces guarantees a brighter future for your apps. Let's dive into Neverendering story of modern front-end architecture.
Node Congress 2024Node Congress 2024
24 min
Optimizing Microservice Architecture for High Performance and Resilience
- Delve into the intricacies of optimizing microservice architecture for achieving high performance and resilience.- Explore the challenges related to performance bottlenecks and resilience in microservices-based systems.- Deep dive into the strategies for enhancing performance, such as efficient communication protocols, asynchronous messaging, and load balancing, while also discussing techniques for building resilience, including circuit breakers, fault tolerance, and chaos engineering.- Explore relevant tooling and technologies, such as service mesh and container orchestration, and offer insightful case studies and lessons learned from real-world implementations.- Emphasize the importance of continuous improvement and adaptation in microservices environments, alongside reflections on the future trajectory of microservices architecture.

Workshops on related topic

Node Congress 2023Node Congress 2023
119 min
Decomposing Monolith NestJS API into GRPC Microservices
The workshop focuses on concepts, algorithms, and practices to decompose a monolithic application into GRPC microservices. It overviews architecture principles, design patterns, and technologies used to build microservices. It covers the theory of the GRPC framework and protocol buffers mechanism, as well as techniques and specifics of building isolated TypeScript services in the Node.js stack. The workshop includes a live use case demo of decomposing an API application into a set of microservices. It fits the best architects, tech leads, and developers who want to learn microservices patterns.
Level: AdvancedPatterns: DDD, MicroservicesTechnologies: GRPC, Protocol Buffers, Node.js, TypeScript, NestJS, Express.js, PostgreSQL, TurborepoExample structure: monorepo configuration, packages configuration, common utilities, demo servicePractical exercise: refactor monolith app
Node Congress 2023Node Congress 2023
102 min
Decoupling in Practice
Deploying decoupled and microservice applications isn't just a problem to be solved on migration day. Moving forward with these architectures depends completely on what your team's workflow experience will look like day-to-day post-migration.
The hardest part of this can often be the number of vendors involved. Some targets are best suited for specific frontend frameworks, while others are more so for CMSs and custom APIs. Unfortunately their assumptions, workflows, APIs, and notions of security can be quite different. While there are certain advantages to relying on a strict contract between apps – where backend and frontend teams work is limited to a single vendor – this isn't always realistic. This could be because you're still experimenting, or simply the size of your organization doesn't allow for this kind of specialization just yet.
In this workshop, you'll have a chance to explore a different, single vendor approach to microservices using Strapi and Next.js as an example. You'll deploy each app individually, establishing a workflow from the start that simplifies customization, introducing new features, investigating performance issues, and even framework interchangeability from the start.
Structure:- Getting started- Overview of Strapi- Overview of workflow- Deploying the project- Switching services- Adding the frontend
Prerequisites:- A trial account created- The CLI installed
DevOps.js Conf 2022DevOps.js Conf 2022
163 min
How to develop, build, and deploy Node.js microservices with Pulumi and Azure DevOps
The workshop gives a practical perspective of key principles needed to develop, build, and maintain a set of microservices in the Node.js stack. It covers specifics of creating isolated TypeScript services using the monorepo approach with lerna and yarn workspaces. The workshop includes an overview and a live exercise to create cloud environment with Pulumi framework and Azure services. The sessions fits the best developers who want to learn and practice build and deploy techniques using Azure stack and Pulumi for Node.js.
JSNation 2023JSNation 2023
117 min
How to Convert Crypto Currencies With GRPC Microservices in Node.js
The workshop overviews key architecture principles, design patterns, and technologies used to build microservices in the Node.js stack. It covers the theory of the GRPC framework and protocol buffers mechanism, as well as techniques and specifics of building isolated services using the monorepo approach with lerna and yarn workspaces, TypeScript. The workshop includes a live practical assignment to create a currency converter application that follows microservices paradigms. It fits the best developers who want to learn and practice GRPC microservices pattern with the Node.js platform.
Prerequistes:- Good understanding of JavaScript or TypeScript- Experience with Node.js and writing Backend applications- Preinstall Node.js, npm- Preinstall Protocol Buffer Compiler- We prefer to use VSCode for a better experience with JavaScript and TypeScript (other IDEs are also ok)
Node Congress 2022Node Congress 2022
162 min
How to Convert Crypto Currencies with Microservices in Node.js and GRPC
The workshop overviews key architecture principles, design patterns, and technologies used to build microservices in the Node.js stack. It covers the theory of the GRPC framework and protocol buffers mechanism, as well as techniques and specifics of building isolated services using the monorepo approach with lerna and yarn workspaces, TypeScript. The workshop includes a live practical assignment to create a currency converter application that follows microservices paradigms. The "Microservices in Node.js with GRPC" workshop fits the best developers who want to learn and practice GRPC microservices pattern with the Node.js platform.