Examining Observability in Node.js

Rate this content
Bookmark

Imagine your productivity and confidence developing web apps without chrome dev tools. Many do exactly that with Node.js.

It is important to observe and learn what’s happening in your app to stay competitive and create the most performant and efficient Node.js applications, following the best practices.

In this talk, we will explore useful tools to examine your Node.js applications and how observability will speed up development, produce better code while improving reliability and uptime. Don’t miss it!

Lizz Parody
Lizz Parody
22 min
24 Jun, 2021

Comments

Sign in or register to post your comment.

Video Summary and Transcription

Today's Talk explores the concept of observability in Node.js, emphasizing the importance of understanding what's happening inside a system without needing to ship new code. The Talk covers various observability tools and techniques, including performance measurement and tracing, HeapSnapshot and Chrome DevTools, the BA Inspector, and external tools like nSolid. nSolid is highlighted as an enhanced observability tool specifically built for Node.js, offering low-input performance insights and greater security for mission-critical applications.

Available in Español

1. Introduction to Observability in Node.js

Short description:

Today, I will be talking about examining observability in Node.js. Observability is a term from control theory. In several products and services, observability means you can answer any questions about what's happening on the inside of the system by observing or by asking questions from the outside of the system without needing to ship new code to answer questions. Because system complexity is outposing our ability to predict what it's going to break. There's a lot of useful tools that come to rescue.

Hello, all. Today, I will be talking about examining observability in Node.js. Hi, my name is Liz Paradis, I'm the head of Developers Relations at Nodesters, and I am from the beautiful country of Colombia.

This is the agenda for today. First, we're going to talk about what is observability, then why observability is important, how we can use Node.js internals for observability, including performance hooks, trace events, heap snapshots, BA native module. And then we will look at some external tools, benchmarks, and conclusions at the end.

So, let's begin by understanding what observability is and why it's important. Observability is a term from control theory. A simple definition could be it's the measure of how well internal states of a system can be inferred by the knowledge of its external outputs. In other words, we see the results, or the output, and we can know what's happening on the inside. Let's take, for example, bananas and avocados. In Colombia, we have a lot of bananas and avocados. It's amazing. And just from looking at the outside, we can know how it is on the inside, if they're ripped, or if they're not ready for consumption yet. In several products and services, observability means you can answer any questions about what's happening on the inside of the system by observing or by asking questions from the outside of the system without needing to ship new code to answer questions. And that's very important. We shouldn't need to write new code to observe what's happening on one of our systems. Because system complexity is outposing our ability to predict what it's going to break. There's a lot of useful tools that come to rescue.

2. Importance of Observability and Choosing Tools

Short description:

Talking about system complexity, software is becoming exponentially more complex. The number of products and tools are multiplying. With environments as complex as we see today, simply monitoring from problems is not enough. Observability gives you the instrumentation you need to understand what's happening in your software and fix it. Observing is exposing the internal state to be viewed externally, and monitoring is collecting and displaying the information that has been exposed. To solve complex nodejs problems, it's necessary to have an observability tool. In choosing an observability tool, we're going to focus on nodejs internals and external tools like performance hooks.

Talking about system complexity, software is becoming exponentially more complex. In infrastructure, we're seeing things such as microservices, containers, Docker, Kubernetes, and others that decompose monoliths into microservices, which are great for products but can be hard on humans. The number of products and tools are also multiplying. There are countless platforms and tools for empowering people to have better observability and control over their code. Great for users. But it's very hard to choose which one is the best one.

Now that you know what observability is, you will be wondering why it's important and why you should care. With environments as complex as we see today, simply monitoring from problems is not enough to recognize, find, and fix a number of issues that arise. Sometimes the new issues are known or not, which means that you don't know the problem, and even worse, you don't know how to find it. So without tools to observe the environment, it's almost impossible to fix that problem. This is why observability is important. It gives you the instrumentation you need to understand what's happening in your software and fix it. A good observability tool helps you find what and where the problem is. It doesn't add overhead to your app. We don't want our app to slow down. Quite the opposite. It also has enhanced security, has flexible integration, and doesn't need to modify your code. And it's also very important to differentiate between observing and monitoring. Observing is exposing the internal state to be viewed externally, and monitoring is collecting and displaying the information that has been exposed, and usually involves writing automation tools around that.

If you want to solve the most complex nodejs problems such as memory leaks, or performance issues, or even just to follow the best practice to keep your code healthy, it's necessary to have an observability tool. The next step will be choosing an observability tool that are best for our needs. So in choosing an observability tool, we're going to focus first on nodejs internals, and then we're going to check some external tools. In nodejs internals for observability, first we're going to talk about performance hooks. This is particularly helpful for checking on performance. It's an object that can be used to collect performance metric from the current nodejs instance. Performance monitoring is not something you should start considering once you start seeing problems. Instead, it should be part of your development process in order to detect possible problems before they are visible in production. Because of the asynchronous nature of nodes, code profiling with regular tools can be very challenging. Especially because part of the time spent could be outside of your own code and inside of the event loop itself. This is why it's important to use even internal nodejs tools like performance hook or external tools as we're going to discuss later.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

It's a Jungle Out There: What's Really Going on Inside Your Node_Modules Folder
Node Congress 2022Node Congress 2022
26 min
It's a Jungle Out There: What's Really Going on Inside Your Node_Modules Folder
Top Content
Do you know what’s really going on in your node_modules folder? Software supply chain attacks have exploded over the past 12 months and they’re only accelerating in 2022 and beyond. We’ll dive into examples of recent supply chain attacks and what concrete steps you can take to protect your team from this emerging threat.
You can check the slides for Feross' talk here.
Towards a Standard Library for JavaScript Runtimes
Node Congress 2022Node Congress 2022
34 min
Towards a Standard Library for JavaScript Runtimes
Top Content
You can check the slides for James' talk here.
ESM Loaders: Enhancing Module Loading in Node.js
JSNation 2023JSNation 2023
22 min
ESM Loaders: Enhancing Module Loading in Node.js
Native ESM support for Node.js was a chance for the Node.js project to release official support for enhancing the module loading experience, to enable use cases such as on the fly transpilation, module stubbing, support for loading modules from HTTP, and monitoring.
While CommonJS has support for all this, it was never officially supported and was done by hacking into the Node.js runtime code. ESM has fixed all this. We will look at the architecture of ESM loading in Node.js, and discuss the loader API that supports enhancing it. We will also look into advanced features such as loader chaining and off thread execution.
Out of the Box Node.js Diagnostics
Node Congress 2022Node Congress 2022
34 min
Out of the Box Node.js Diagnostics
In the early years of Node.js, diagnostics and debugging were considerable pain points. Modern versions of Node have improved considerably in these areas. Features like async stack traces, heap snapshots, and CPU profiling no longer require third party modules or modifications to application source code. This talk explores the various diagnostic features that have recently been built into Node.
You can check the slides for Colin's talk here. 
Node.js Compatibility in Deno
Node Congress 2022Node Congress 2022
34 min
Node.js Compatibility in Deno
Can Deno run apps and libraries authored for Node.js? What are the tradeoffs? How does it work? What’s next?
Multithreaded Logging with Pino
JSNation Live 2021JSNation Live 2021
19 min
Multithreaded Logging with Pino
Top Content
Almost every developer thinks that adding one more log line would not decrease the performance of their server... until logging becomes the biggest bottleneck for their systems! We created one of the fastest JSON loggers for Node.js: pino. One of our key decisions was to remove all "transport" to another process (or infrastructure): it reduced both CPU and memory consumption, removing any bottleneck from logging. However, this created friction and lowered the developer experience of using Pino and in-process transports is the most asked feature our user.In the upcoming version 7, we will solve this problem and increase throughput at the same time: we are introducing pino.transport() to start a worker thread that you can use to transfer your logs safely to other destinations, without sacrificing neither performance nor the developer experience.

Workshops on related topic

Node.js Masterclass
Node Congress 2023Node Congress 2023
109 min
Node.js Masterclass
Top Content
Workshop
Matteo Collina
Matteo Collina
Have you ever struggled with designing and structuring your Node.js applications? Building applications that are well organised, testable and extendable is not always easy. It can often turn out to be a lot more complicated than you expect it to be. In this live event Matteo will show you how he builds Node.js applications from scratch. You’ll learn how he approaches application design, and the philosophies that he applies to create modular, maintainable and effective applications.

Level: intermediate
Build and Deploy a Backend With Fastify & Platformatic
JSNation 2023JSNation 2023
104 min
Build and Deploy a Backend With Fastify & Platformatic
WorkshopFree
Matteo Collina
Matteo Collina
Platformatic allows you to rapidly develop GraphQL and REST APIs with minimal effort. The best part is that it also allows you to unleash the full potential of Node.js and Fastify whenever you need to. You can fully customise a Platformatic application by writing your own additional features and plugins. In the workshop, we’ll cover both our Open Source modules and our Cloud offering:- Platformatic OSS (open-source software) — Tools and libraries for rapidly building robust applications with Node.js (https://oss.platformatic.dev/).- Platformatic Cloud (currently in beta) — Our hosting platform that includes features such as preview apps, built-in metrics and integration with your Git flow (https://platformatic.dev/). 
In this workshop you'll learn how to develop APIs with Fastify and deploy them to the Platformatic Cloud.
0 to Auth in an Hour Using NodeJS SDK
Node Congress 2023Node Congress 2023
63 min
0 to Auth in an Hour Using NodeJS SDK
WorkshopFree
Asaf Shen
Asaf Shen
Passwordless authentication may seem complex, but it is simple to add it to any app using the right tool.
We will enhance a full-stack JS application (Node.JS backend + React frontend) to authenticate users with OAuth (social login) and One Time Passwords (email), including:- User authentication - Managing user interactions, returning session / refresh JWTs- Session management and validation - Storing the session for subsequent client requests, validating / refreshing sessions
At the end of the workshop, we will also touch on another approach to code authentication using frontend Descope Flows (drag-and-drop workflows), while keeping only session validation in the backend. With this, we will also show how easy it is to enable biometrics and other passwordless authentication methods.
Table of contents- A quick intro to core authentication concepts- Coding- Why passwordless matters
Prerequisites- IDE for your choice- Node 18 or higher
Building a Hyper Fast Web Server with Deno
JSNation Live 2021JSNation Live 2021
156 min
Building a Hyper Fast Web Server with Deno
WorkshopFree
Matt Landers
Will Johnston
2 authors
Deno 1.9 introduced a new web server API that takes advantage of Hyper, a fast and correct HTTP implementation for Rust. Using this API instead of the std/http implementation increases performance and provides support for HTTP2. In this workshop, learn how to create a web server utilizing Hyper under the hood and boost the performance for your web apps.
GraphQL - From Zero to Hero in 3 hours
React Summit 2022React Summit 2022
164 min
GraphQL - From Zero to Hero in 3 hours
Workshop
Pawel Sawicki
Pawel Sawicki
How to build a fullstack GraphQL application (Postgres + NestJs + React) in the shortest time possible.
All beginnings are hard. Even harder than choosing the technology is often developing a suitable architecture. Especially when it comes to GraphQL.
In this workshop, you will get a variety of best practices that you would normally have to work through over a number of projects - all in just three hours.
If you've always wanted to participate in a hackathon to get something up and running in the shortest amount of time - then take an active part in this workshop, and participate in the thought processes of the trainer.
Mastering Node.js Test Runner
TestJS Summit 2023TestJS Summit 2023
78 min
Mastering Node.js Test Runner
Workshop
Marco Ippolito
Marco Ippolito
Node.js test runner is modern, fast, and doesn't require additional libraries, but understanding and using it well can be tricky. You will learn how to use Node.js test runner to its full potential. We'll show you how it compares to other tools, how to set it up, and how to run your tests effectively. During the workshop, we'll do exercises to help you get comfortable with filtering, using native assertions, running tests in parallel, using CLI, and more. We'll also talk about working with TypeScript, making custom reports, and code coverage.