Monitoring, Alerting, And Visualizing your Node.JS server infrastructure with Open Source tools

Rate this content
Bookmark

When monitoring Node.js, you can track your applications’ performance and availability by finding bottlenecks and fixing errors. You can identify issues by specifically looking at metrics like Process memory usage, Average response time, CPU usage, and more. If you add monitoring of the other components of your entire stack, you will gain a comprehensive view of what could be impacting application performance.  At that point, it can point out the problem at the code level — allowing you to track down and fix those issues before they negatively impact end user experience.  This talk will focus on the tools available from the Open Source time series database InfluxDB. It will be using the open source node.js telegraf plugin, so you can easily collect key metrics to help you get that view into your application. We will be using the Node.js Monitoring Template which is prebuilt and equipped to monitor an applications' performance and availability. All code examples will be in Javascript, and we will also go over the javascript library for those who are working in other javascript server environments, or who want to export data to their preferred visualization tools. 

Zoe Steinkamp
Zoe Steinkamp
31 min
14 Apr, 2023

Comments

Sign in or register to post your comment.

Video Summary and Transcription

This Talk introduces monitoring, alerting, and visualizing Node.js server infrastructure with open source tools. It covers the use of time series databases for server monitoring and discusses the JavaScript client library. The InfluxDB platform offers features for data acquisition, visualization, and task scheduling. The Talk includes a live demo of a fake Express server and showcases the use of community templates and the Node.js dashboard. It also provides resources for further learning and discusses options for visualization with Grafana.

1. Introduction to Node.js Server Monitoring

Short description:

I'm going to be going over monitoring, alerting, and visualizing your Node.js server infrastructure with open source tooling. We're going to really quickly go over why you want to use a time series database when it comes to your server monitoring. We're going to briefly talk about the JavaScript client library. We're going to talk about this awesome monitoring template that we offer for Node.js server monitoring that you can easily install and get going on. I'm going to do a live demo where I kind of show off all of these pieces all put together. Finally, Q and A and further resources.

Awesome. Thank you so much for the introduction. I have to admit, the video kind of startled me, I didn't expect the noise. So we're going to go ahead and get started here. I'm going to be going over monitoring, alerting, and visualizing your Node.js server infrastructure with open source tooling.

So let's go ahead. We already got a pretty good introduction. But if you guys want to add me on LinkedIn, if you don't want to ask questions here, which I don't know why you wouldn't, because it's all very, what's the word, you can just ask them and not have to attach your name to them. Yeah, so it's really quite great. But otherwise, you're welcome to connect with me there.

So let's go over really quick the session agenda, because we only have about 20 minutes here. So we're going to really quickly go over why you want to use a time series database when it comes to your server monitoring. We're going to really quick talk over influx data and platform tools that it comes built in with. So influx data is a fully open source, influx DB time series database. We also do offer a cloud, but we're just going to be talking about the open source during this talk. We're going to briefly talk about the JavaScript client library. We're going to talk about this awesome monitoring template that we offer for Node.js server monitoring that you can easily install and get going on. I'm going to do a live demo where I kind of show off all of these pieces all put together. The reason I kind of put some code inside the slides and such is because otherwise I might forget to show them to you and it might not make quite as much sense. But we're going to try and give some extra time to that demo. And then finally, Q and A and further resources. So I will have like a final slide deck that will have a ton of the links that you can take a photo of with your phone. Because I know people tend to take photos throughout. So I do put them all at the end for you guys for more convenience. So the importance of time series databases. So time series data is any data that you want to deal with over a timespan. Now obviously right now we're talking about server infrastructure monitoring, which is very much time based. It's all about what happened at this time of the day, when did the server scale up, when did it scale down, when did it stop giving 200 responses and start giving four hundreds instead. These are all things that happen over the span of time. Now the other thing is though, when we talk about time series, we also normally tend to talk about more physical things.

2. Time Series Applications and Server Monitoring

Short description:

You can think of your Fitbit or Health Tracker. That's a time series device that you just happen to wear. Nowadays, server monitoring has become a lot more important. There's plenty of databases that you can choose from, but for the most part, we see other DBs as good for other types of data. Let's really quick go over this overview so we kind of get an idea of some of the tools that we can start using. Inside the InfluxDB platform, we have Telegraf, which is fully open source. Today we're going to go over the JavaScript/Node.js client library.

You can think of your Fitbit or Health Tracker. That's a time series device that you just happen to wear. Your thermostat at home is most likely streaming time series data somewhere, solar panels on your house. Nowadays in this world we have a lot more time series applications, and nowadays also server monitoring has become a lot more important.

They say the average user will only wait about three seconds for a website to load. I swear I only wait two. We expect everything to work perfectly, and we don't like when things are down. Nowadays, server monitoring has become a lot more important.

One thing to take note, which I already kind of explained here, but that doesn't mean that you can't use other DBs for your server monitoring data. That's not the case at all. There's plenty of databases that you can choose from, but for the most part, we see other DBs as good for other types of data.

This is just a few listed examples here. As you guys know, you're probably using Elastic if you have any searching on your website. You might use Mongo for documents. Of course, we all use SQL for, I would say, about 90% of data in the World Wide Web is actually stored in, yeah, a SQL-type DB. These are all DBs that offer some form of open source as well.

Let's really quick go over this overview so we kind of get an idea of some of the tools that we can start using. This is what the platform ends up looking like. We have those data sources, which I've already mentioned. For us, it's going to be things like cloud services most likely. And then inside the InfluxDB platform, we have Telegraf, which some of you guys might actually have used before in the past. It's just not necessarily known that InfluxDB is the caretaker of Telegraf. It's fully open source. We don't write all the code for it. A lot of the companies or individuals write the libraries themselves. But we tend to take care of it in that we don't allow malicious code uploads and such. The other thing is we have our client libraries. Those are managed by us. Today we're going to kind of go over the JavaScript slash Node.js client library. They are combined inside of the client library, but they do have slightly different functionalities.

QnA

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

It's a Jungle Out There: What's Really Going on Inside Your Node_Modules Folder
Node Congress 2022Node Congress 2022
26 min
It's a Jungle Out There: What's Really Going on Inside Your Node_Modules Folder
Top Content
Do you know what’s really going on in your node_modules folder? Software supply chain attacks have exploded over the past 12 months and they’re only accelerating in 2022 and beyond. We’ll dive into examples of recent supply chain attacks and what concrete steps you can take to protect your team from this emerging threat.
You can check the slides for Feross' talk here.
Towards a Standard Library for JavaScript Runtimes
Node Congress 2022Node Congress 2022
34 min
Towards a Standard Library for JavaScript Runtimes
Top Content
You can check the slides for James' talk here.
ESM Loaders: Enhancing Module Loading in Node.js
JSNation 2023JSNation 2023
22 min
ESM Loaders: Enhancing Module Loading in Node.js
Native ESM support for Node.js was a chance for the Node.js project to release official support for enhancing the module loading experience, to enable use cases such as on the fly transpilation, module stubbing, support for loading modules from HTTP, and monitoring.
While CommonJS has support for all this, it was never officially supported and was done by hacking into the Node.js runtime code. ESM has fixed all this. We will look at the architecture of ESM loading in Node.js, and discuss the loader API that supports enhancing it. We will also look into advanced features such as loader chaining and off thread execution.
Out of the Box Node.js Diagnostics
Node Congress 2022Node Congress 2022
34 min
Out of the Box Node.js Diagnostics
In the early years of Node.js, diagnostics and debugging were considerable pain points. Modern versions of Node have improved considerably in these areas. Features like async stack traces, heap snapshots, and CPU profiling no longer require third party modules or modifications to application source code. This talk explores the various diagnostic features that have recently been built into Node.
You can check the slides for Colin's talk here. 
Node.js Compatibility in Deno
Node Congress 2022Node Congress 2022
34 min
Node.js Compatibility in Deno
Can Deno run apps and libraries authored for Node.js? What are the tradeoffs? How does it work? What’s next?
Multithreaded Logging with Pino
JSNation Live 2021JSNation Live 2021
19 min
Multithreaded Logging with Pino
Top Content
Almost every developer thinks that adding one more log line would not decrease the performance of their server... until logging becomes the biggest bottleneck for their systems! We created one of the fastest JSON loggers for Node.js: pino. One of our key decisions was to remove all "transport" to another process (or infrastructure): it reduced both CPU and memory consumption, removing any bottleneck from logging. However, this created friction and lowered the developer experience of using Pino and in-process transports is the most asked feature our user.In the upcoming version 7, we will solve this problem and increase throughput at the same time: we are introducing pino.transport() to start a worker thread that you can use to transfer your logs safely to other destinations, without sacrificing neither performance nor the developer experience.

Workshops on related topic

Node.js Masterclass
Node Congress 2023Node Congress 2023
109 min
Node.js Masterclass
Top Content
Workshop
Matteo Collina
Matteo Collina
Have you ever struggled with designing and structuring your Node.js applications? Building applications that are well organised, testable and extendable is not always easy. It can often turn out to be a lot more complicated than you expect it to be. In this live event Matteo will show you how he builds Node.js applications from scratch. You’ll learn how he approaches application design, and the philosophies that he applies to create modular, maintainable and effective applications.

Level: intermediate
Build and Deploy a Backend With Fastify & Platformatic
JSNation 2023JSNation 2023
104 min
Build and Deploy a Backend With Fastify & Platformatic
WorkshopFree
Matteo Collina
Matteo Collina
Platformatic allows you to rapidly develop GraphQL and REST APIs with minimal effort. The best part is that it also allows you to unleash the full potential of Node.js and Fastify whenever you need to. You can fully customise a Platformatic application by writing your own additional features and plugins. In the workshop, we’ll cover both our Open Source modules and our Cloud offering:- Platformatic OSS (open-source software) — Tools and libraries for rapidly building robust applications with Node.js (https://oss.platformatic.dev/).- Platformatic Cloud (currently in beta) — Our hosting platform that includes features such as preview apps, built-in metrics and integration with your Git flow (https://platformatic.dev/). 
In this workshop you'll learn how to develop APIs with Fastify and deploy them to the Platformatic Cloud.
0 to Auth in an Hour Using NodeJS SDK
Node Congress 2023Node Congress 2023
63 min
0 to Auth in an Hour Using NodeJS SDK
WorkshopFree
Asaf Shen
Asaf Shen
Passwordless authentication may seem complex, but it is simple to add it to any app using the right tool.
We will enhance a full-stack JS application (Node.JS backend + React frontend) to authenticate users with OAuth (social login) and One Time Passwords (email), including:- User authentication - Managing user interactions, returning session / refresh JWTs- Session management and validation - Storing the session for subsequent client requests, validating / refreshing sessions
At the end of the workshop, we will also touch on another approach to code authentication using frontend Descope Flows (drag-and-drop workflows), while keeping only session validation in the backend. With this, we will also show how easy it is to enable biometrics and other passwordless authentication methods.
Table of contents- A quick intro to core authentication concepts- Coding- Why passwordless matters
Prerequisites- IDE for your choice- Node 18 or higher
Building a Hyper Fast Web Server with Deno
JSNation Live 2021JSNation Live 2021
156 min
Building a Hyper Fast Web Server with Deno
WorkshopFree
Matt Landers
Will Johnston
2 authors
Deno 1.9 introduced a new web server API that takes advantage of Hyper, a fast and correct HTTP implementation for Rust. Using this API instead of the std/http implementation increases performance and provides support for HTTP2. In this workshop, learn how to create a web server utilizing Hyper under the hood and boost the performance for your web apps.
GraphQL - From Zero to Hero in 3 hours
React Summit 2022React Summit 2022
164 min
GraphQL - From Zero to Hero in 3 hours
Workshop
Pawel Sawicki
Pawel Sawicki
How to build a fullstack GraphQL application (Postgres + NestJs + React) in the shortest time possible.
All beginnings are hard. Even harder than choosing the technology is often developing a suitable architecture. Especially when it comes to GraphQL.
In this workshop, you will get a variety of best practices that you would normally have to work through over a number of projects - all in just three hours.
If you've always wanted to participate in a hackathon to get something up and running in the shortest amount of time - then take an active part in this workshop, and participate in the thought processes of the trainer.
Mastering Node.js Test Runner
TestJS Summit 2023TestJS Summit 2023
78 min
Mastering Node.js Test Runner
Workshop
Marco Ippolito
Marco Ippolito
Node.js test runner is modern, fast, and doesn't require additional libraries, but understanding and using it well can be tricky. You will learn how to use Node.js test runner to its full potential. We'll show you how it compares to other tools, how to set it up, and how to run your tests effectively. During the workshop, we'll do exercises to help you get comfortable with filtering, using native assertions, running tests in parallel, using CLI, and more. We'll also talk about working with TypeScript, making custom reports, and code coverage.