Next Generation Code Architecture for Building Maintainable Node Applications

Rate this content
Bookmark

In today's fast-paced software development landscape, it's essential to have tools that allow us to build, test, and deploy our applications quickly and efficiently. Being able to ship features fast implies having a healthy and maintainable codebase, which can be tricky and daunting, especially in the long-run.

In this talk, we'll explore strategies for building maintainable Node backends by leveraging tooling that Nx provides. This includes how to modularize a codebase, using code generators for consistency, establish code boundaries, and how to keep CI fast as your codebase grows.


Juri Strumpflohner
Juri Strumpflohner
30 min
14 Apr, 2023

Comments

Sign in or register to post your comment.

Video Summary and Transcription

Today's Talk focused on code architecture, modularization, and scaling in software development. The speaker discussed the benefits of separating code by domain and using tools like NX to improve productivity and enforce modular architecture. They also highlighted the importance of automating library creation and configuration. Additionally, the Talk covered code scaling and deployment strategies, including caching and automated code migrations. The speaker emphasized the flexibility and scalability of Fastify and the advantages of using a monorepo for front-end and back-end development.

Available in Español

1. Code Architecture and Scaling

Short description:

Today, we will discuss code architecture and building maintainable node applications from a tooling perspective. We often see a problem with scattered features across different folders, which hinders scalability and causes merge conflicts. A better approach is separation by domain, which allows for atomic and localized features. We will also explore domain modules, automation, and code scaling as the product grows.

So let's dive right in! Quite a mouthful of a title actually, but what I would want to look a bit into today is kind of the code architecture and building maintainable node applications, but from a tooling perspective.

The main reason is, independently of front-end or back-end projects, which I've seen as part of my consulting, as part of working with some of the clients, I often see a structure like this, which is perfectly fine when you start a new project, but the main problem here that you see is if I'm talking about adding features to products, I'm kind of having it scattered across those different folders based on the structure that I have here.

And the thing is because this is a separation by type. So we have all APIs that are REST or whatever we're using, maybe TRPC, are in that API layer while then the services are in the service layer and the data access in the data access layer.

And the project doesn't really scale. As you add more features, you will not just have one file, as in this very, very simple case here, an example, but also if you add new team members, they can constantly work across these folders, and it's very easy that they get into problems like merge conflicts, stuff like that.

So there's an alternative for that, which is separation by domain. I'm pretty sure you've seen this. A lot of people actually do this. I think that that's a better approach there, simply because now you can sort the different features out in their own areas to become more atomic, more localized into one single area of your entire product.

And again, you get the benefits out of that. And these are the things that I would want to touch a bit and talk today. So talking a bit about domain modules, how we can structure that, how we can add in automation to help us with that and make sure that we stay within those domain modules. And then also a bit about code scaling in the sense of what happens if I add more of these and keep adding, maybe add a Monorepo, stuff like that. So how can I make sure that my code scales as my product gets bigger?

2. Introduction to NX and Modular Architecture

Short description:

I'm currently the senior director of developer experience for NX, Google developer expert and AI instructor. NX is open source and helps improve developer productivity. It can be used in both Monorepo and single project setups. Modularizing by domain boundaries improves maintainability, flexibility, reusability, and testability. However, importing modules from different domains can still happen accidentally. NX addresses this issue by providing guardrails and modular architecture. The base layer of NX includes the workspace, while plugins offer technology-specific tooling automation. Standalone product was introduced to repurpose the structure of NX.

I'm currently the senior director of developer experience for NX, Google developer expert and AI instructor. NX is open source, and it is a tool for kind of like helping you improve developer productivity. So there is a set of tools and techniques, you can incrementally adopt it, like on the lower level, and then add more stuff on top of it. We're kind of known for Monorepos, but today I'm actually more about talking about the standalone product side of it. So you don't just can use NX in a Monorepo, but it's actually also useful for single project.

So why modularized by domain boundaries? Nowadays, you almost need to ask chatGPD just to make sure you're on the right track, but I actually came up with some good answers there. And especially, obviously, maintainability part of it, right? Because as we mentioned before, you have those small, more cohesive features, those modules are nicely encapsulated. The flexibility, because you can go ahead and potentially rip out a module, because it's kind of out of consistent, not always that easy and reusability. As you start splitting up, like, you might see patterns of things that are reused that are kind of similar across domains. So you can extract them even more and kind of then actually reuse them in your code. Testability is also a nice side effect. And there are more of these types of things, because now that you have modules, you can potentially just test that single thing in isolation as well.

So what though prevents me from doing something like this, right? Because now I have my nice structure, structured by modules, domain driven development kind of approach. But nothing actually prevents me from just importing, let's say, here my order service imports something from the product list API because someone has a function in their node file layer and their node API, and I'm just importing it. Maybe even not intentionally, just my idea auto-completes and pulls in that utility function. Can we do better? Like, can we have something in place to actually restrict that a bit more? And this is where we started thinking about at NX quite a lot. Because, as in the intro was mentioned, we actually did consulting for quite large companies, which have usually large code base. They run into these type of issues continuously. Because they have like 60 to 300 developers on that same code base and they keep adding features whole day, right? So you want to have some guardrails in place. Now, as mentioned, NX is known for the monorepo kind of thing. But if you look at the architecture of NX, it is actually Modular itself. So at the base layer, you can see at the very top there, there is your workspace. That can directly just use the base layer of NX, which means you get the task running, you get some caching, which is useful for monorepos. But then on top, optionally, you can also have these plugins. And plugins are, you can imagine them like technology-specific tooling automation that make your life easier. So they can generate code, they can abstract some of the lower-level build tuning, provide code migrations, all that sort of things. And these plugins specifically are actually very interesting in a single product setup as well. These code generation doesn't have to do anything really with monorepos. It's just like an ergonomic tool that can use to make your life easier. So standalone product was introduced a couple of months ago, it was well almost a half year ago, where you kind of just repurposed how the structure of NX looks like.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

It's a Jungle Out There: What's Really Going on Inside Your Node_Modules Folder
Node Congress 2022Node Congress 2022
26 min
It's a Jungle Out There: What's Really Going on Inside Your Node_Modules Folder
Top Content
Do you know what’s really going on in your node_modules folder? Software supply chain attacks have exploded over the past 12 months and they’re only accelerating in 2022 and beyond. We’ll dive into examples of recent supply chain attacks and what concrete steps you can take to protect your team from this emerging threat.
You can check the slides for Feross' talk here.
Towards a Standard Library for JavaScript Runtimes
Node Congress 2022Node Congress 2022
34 min
Towards a Standard Library for JavaScript Runtimes
Top Content
You can check the slides for James' talk here.
ESM Loaders: Enhancing Module Loading in Node.js
JSNation 2023JSNation 2023
22 min
ESM Loaders: Enhancing Module Loading in Node.js
Native ESM support for Node.js was a chance for the Node.js project to release official support for enhancing the module loading experience, to enable use cases such as on the fly transpilation, module stubbing, support for loading modules from HTTP, and monitoring.
While CommonJS has support for all this, it was never officially supported and was done by hacking into the Node.js runtime code. ESM has fixed all this. We will look at the architecture of ESM loading in Node.js, and discuss the loader API that supports enhancing it. We will also look into advanced features such as loader chaining and off thread execution.
Out of the Box Node.js Diagnostics
Node Congress 2022Node Congress 2022
34 min
Out of the Box Node.js Diagnostics
In the early years of Node.js, diagnostics and debugging were considerable pain points. Modern versions of Node have improved considerably in these areas. Features like async stack traces, heap snapshots, and CPU profiling no longer require third party modules or modifications to application source code. This talk explores the various diagnostic features that have recently been built into Node.
You can check the slides for Colin's talk here. 
Node.js Compatibility in Deno
Node Congress 2022Node Congress 2022
34 min
Node.js Compatibility in Deno
Can Deno run apps and libraries authored for Node.js? What are the tradeoffs? How does it work? What’s next?
Multithreaded Logging with Pino
JSNation Live 2021JSNation Live 2021
19 min
Multithreaded Logging with Pino
Top Content
Almost every developer thinks that adding one more log line would not decrease the performance of their server... until logging becomes the biggest bottleneck for their systems! We created one of the fastest JSON loggers for Node.js: pino. One of our key decisions was to remove all "transport" to another process (or infrastructure): it reduced both CPU and memory consumption, removing any bottleneck from logging. However, this created friction and lowered the developer experience of using Pino and in-process transports is the most asked feature our user.In the upcoming version 7, we will solve this problem and increase throughput at the same time: we are introducing pino.transport() to start a worker thread that you can use to transfer your logs safely to other destinations, without sacrificing neither performance nor the developer experience.

Workshops on related topic

Node.js Masterclass
Node Congress 2023Node Congress 2023
109 min
Node.js Masterclass
Top Content
Workshop
Matteo Collina
Matteo Collina
Have you ever struggled with designing and structuring your Node.js applications? Building applications that are well organised, testable and extendable is not always easy. It can often turn out to be a lot more complicated than you expect it to be. In this live event Matteo will show you how he builds Node.js applications from scratch. You’ll learn how he approaches application design, and the philosophies that he applies to create modular, maintainable and effective applications.

Level: intermediate
Build and Deploy a Backend With Fastify & Platformatic
JSNation 2023JSNation 2023
104 min
Build and Deploy a Backend With Fastify & Platformatic
WorkshopFree
Matteo Collina
Matteo Collina
Platformatic allows you to rapidly develop GraphQL and REST APIs with minimal effort. The best part is that it also allows you to unleash the full potential of Node.js and Fastify whenever you need to. You can fully customise a Platformatic application by writing your own additional features and plugins. In the workshop, we’ll cover both our Open Source modules and our Cloud offering:- Platformatic OSS (open-source software) — Tools and libraries for rapidly building robust applications with Node.js (https://oss.platformatic.dev/).- Platformatic Cloud (currently in beta) — Our hosting platform that includes features such as preview apps, built-in metrics and integration with your Git flow (https://platformatic.dev/). 
In this workshop you'll learn how to develop APIs with Fastify and deploy them to the Platformatic Cloud.
0 to Auth in an Hour Using NodeJS SDK
Node Congress 2023Node Congress 2023
63 min
0 to Auth in an Hour Using NodeJS SDK
WorkshopFree
Asaf Shen
Asaf Shen
Passwordless authentication may seem complex, but it is simple to add it to any app using the right tool.
We will enhance a full-stack JS application (Node.JS backend + React frontend) to authenticate users with OAuth (social login) and One Time Passwords (email), including:- User authentication - Managing user interactions, returning session / refresh JWTs- Session management and validation - Storing the session for subsequent client requests, validating / refreshing sessions
At the end of the workshop, we will also touch on another approach to code authentication using frontend Descope Flows (drag-and-drop workflows), while keeping only session validation in the backend. With this, we will also show how easy it is to enable biometrics and other passwordless authentication methods.
Table of contents- A quick intro to core authentication concepts- Coding- Why passwordless matters
Prerequisites- IDE for your choice- Node 18 or higher
Building a Hyper Fast Web Server with Deno
JSNation Live 2021JSNation Live 2021
156 min
Building a Hyper Fast Web Server with Deno
WorkshopFree
Matt Landers
Will Johnston
2 authors
Deno 1.9 introduced a new web server API that takes advantage of Hyper, a fast and correct HTTP implementation for Rust. Using this API instead of the std/http implementation increases performance and provides support for HTTP2. In this workshop, learn how to create a web server utilizing Hyper under the hood and boost the performance for your web apps.
GraphQL - From Zero to Hero in 3 hours
React Summit 2022React Summit 2022
164 min
GraphQL - From Zero to Hero in 3 hours
Workshop
Pawel Sawicki
Pawel Sawicki
How to build a fullstack GraphQL application (Postgres + NestJs + React) in the shortest time possible.
All beginnings are hard. Even harder than choosing the technology is often developing a suitable architecture. Especially when it comes to GraphQL.
In this workshop, you will get a variety of best practices that you would normally have to work through over a number of projects - all in just three hours.
If you've always wanted to participate in a hackathon to get something up and running in the shortest amount of time - then take an active part in this workshop, and participate in the thought processes of the trainer.
Mastering Node.js Test Runner
TestJS Summit 2023TestJS Summit 2023
78 min
Mastering Node.js Test Runner
Workshop
Marco Ippolito
Marco Ippolito
Node.js test runner is modern, fast, and doesn't require additional libraries, but understanding and using it well can be tricky. You will learn how to use Node.js test runner to its full potential. We'll show you how it compares to other tools, how to set it up, and how to run your tests effectively. During the workshop, we'll do exercises to help you get comfortable with filtering, using native assertions, running tests in parallel, using CLI, and more. We'll also talk about working with TypeScript, making custom reports, and code coverage.