Machine Learning in Node.js using Tensorflow.js

Rate this content
Bookmark

Write and deploy machine learning models easily in Nodejs using Tensorflow.js.

8 min
24 Jun, 2021

Video Summary and Transcription

The Talk introduces TensorFlow.js in Node.js for machine learning, highlighting its open-source nature and easy integration with JavaScript. It emphasizes the benefits of using Node.js, such as the ability to write machine learning models directly in JavaScript, access to the NPM ecosystem, and improved performance. The different packages available for utilizing TensorFlow.js in Node.js, including CPU, GPU, and vanilla packages, are discussed. The importance of setting up Node.js bindings to avoid blocking the main thread is mentioned, along with the availability of APIs like dfNode and TensorBoard.

1. Introduction to TensorFlow.js in Node.js

Short description:

Hello, everyone. The topic of my lightning talk is machine learning in Node.js using TensorFlow.js. TensorFlow.js is an open-source library that enables easy integration of machine learning models directly in JavaScript. It allows easy access to machine learning applications on different platforms without any need for plugins. The architecture of TensorFlow.js includes pre-trained models, a layers API for building and training models, and an ops or core API for fine-tuned control. Utilizing Node.js to run TensorFlow.js provides the ability to write machine learning models directly in JavaScript, run larger models, utilize the NPM ecosystem, and achieve better performance.

Hello, everyone. The topic of my lightning talk is machine learning in Node.js using TensorFlow.js.

Hi, I'm Shivailamba. I'm currently a TensorFlow.js SIG member and also a Google Codename Mentor at TensorFlow.

So, the first question that comes to everyone's mind is what exactly is TensorFlow.js? So, TensorFlow.js is an open-source library that enables easy integration of machine learning models directly in JavaScript. Hence, it reduces the need to learn a separate language like Python to host machine learning models. And it allows very easy integration of some kind of pre-trained models or writing your own models from scratch directly in JavaScript.

And as we know that JavaScript is a really versatile language, it can be run across different platforms like on the browser, on the mobile, or let's say on even IoT devices running Raspberry Pi. So, all of these different platforms now get easy access to machine learning based applications without any need of plugins, right? And machine learning that can be used in TensorFlow.js allows not just preexisting models, but it can also use a transfer learning to retrain some of the existing models based on your own data set. And you can actually also write machine learning models directly in JavaScript using JavaScript and without the need of any other language like Python.

Now this is the architecture of TensorFlow.js. There are three different APIs that are provided. So the top layer that you see are the TensorFlow.js pre trained models that are completely ready to use JavaScript classes. The next one is the layers API that enables to easily build and train models using high level building blocks similarly to how Keras has been built on top of TensorFlow. And finally we have the ops or the core API that helps to give you fine tuned control of model architecture or let's say for doing mathematical calculations like linear algebra. And this can be run either on the client side and on the server side. So on the client side we have basically the browser or let's say the mobile based applications that use the CPU or let's say the web accelerated graphics that is web GL or WebAssembly. And on the server side there is support for TensorFlow CPU that are run via the C bindings and we also have the support for TensorFlow GPU that help get acceleration with the help of CUDA that is also being run on Python based machine learning models.

And finally, you know, what are some of the main advantages of utilizing node.js to actually run TensorFlow.js. So the first one is, you know, we can write the machine learning models directly from scratch in JavaScript and we are only coding in one language. There's no need to use any other language to run and train and also test out these machine learning models. Now by using the TensorFlow.js on node.js we are also giving the ability to run much larger models that require a lot more power. For example, let's say if you're running some kind of neural networks that are having a lot of epochs and will require graphics based acceleration. So, with the help of the server hardware, like in Python, we can utilize the power of these servers to accelerate our larger models that cannot be done in the client side. And also we have the support for the NPM ecosystem because the NPM ecosystem is such a large like the node modules are so heavily used and are there. So, a wide variety of node models can be used directly with your TensorFlow.js based code as well to help assist with your TensorFlow.js application. And most importantly, we get much better performance because Node.js uses the just-in-time compiler and the models that are written in TensorFlow.js utilizing the Node.js can also get this performance boost. And that is quite evident with a few examples. For example, this chart shows you the comparison of TensorFlow running on Python-based models and also on the TensorFlow.js for a mobile net, which is a convolutional neural network model. And as you can see that the comparison for the time that it actually takes for the model to actually run is not a lot different.

2. Utilizing TensorFlow.js in Node.js

Short description:

TensorFlow.js provides better performance than Python versions in certain cases, such as BERT classification. TensorFlow.js in Node.js can be utilized through three different packages: TensorFlow CPU, TensorFlow GPU, and vanilla package. The CPU package accelerates mathematical computations, while the GPU package runs tensor operations on the GPU for even better performance. The vanilla package, which does not rely on TensorFlow, can be used on other devices that support Node.js. Node.js bindings for TensorFlow.js should be set up to avoid blocking the main thread. APIs, such as dfNode and TensorBoard, are available once the package is imported. Feel free to connect with me on social platforms for any queries regarding TensorFlow.js.

So that means that TensorFlow.js itself is quite optimized for running very industrial standard models as well, and also for newer models at the same time. And in some cases, the TensorFlow.js model actually provides a lot more better performance as compared to the Python versions. For example, BERT, which is a state-of-the-art language model for natural language processing, we can see that over here, there is actually a two times better performance boost by actually using Node.js as compared to a Python-based model that is running this BERT classification.

Now, coming on to the most important part, that is, you know, how can we start utilizing the TensorFlow.js in Node.js. So we get namely three different packages that you can install, like the npm packages. So the first one is the TensorFlow CPU. The TensorFlow CPU is, you know, whenever we are importing this package, the module that we get is basically accelerated by the TensorFlow C binary and it runs on the CPU. And the TensorFlow on the CPU uses the hardware acceleration to, let's say, accelerate any kind of mathematical computations, for example, linear algebra.

Now, the extension to that is the TensorFlow GPU package. Like the CPU package, this particular GPU package will be accelerated via the TensorFlow C binary. But it also runs the tensor operations on the GPU with the help of CUDA, that heavily accelerates your performance. And this binding is definitely the fastest as compared to any other package that we have. And now finally, we also have the vanilla package. This package is very similar to the one that we run on the browser. And in this package, the operations are run in vanilla JavaScript and on the CPU. Now, as compared to the other modules, that is the TensorFlow CPU and the TensorFlow GPU, this does not have support for the TensorFlow binary, which actually makes the overall package very smaller. And because of this, since it does not rely on TensorFlow, it can also be actually used on a lot of other devices that support Node.js.

Now, one of the important considerations that need to be made is that the Node.js bindings run on the backend for TensorFlow.js that implements them synchronously. That means that whenever we are running the Node.js bindings on a production application like a web server, we should actually set up a drop queue or a server or like worker threads so that it does not block your main thread. Now, there are also support for APIs because once we have imported the package, all of the normal TensorFlow.js symbols that we use can be appear once we have imported the specific module. Like, for example, one of them is like the dfNode that contains Node.js specific APIs and TensorBoard is actually a notable example of the Node.js specific APIs.

Now, this is a sample code that should give you an example where we have defined like a model and we are training it by utilizing the tf.node.tensorboard. That helps in your model training. With that, that finishes off my presentation and I hope that you have liked it and you can connect with me on these social platforms and to ask any questions regarding TensorFlow.js. Again, do follow any kind of TensorFlow.js models using the hashtag madewithTensorFlow.js on Twitter and on LinkedIn. And again, feel free to connect with me for any queries regarding TensorFlow.js. I hope you have liked it.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Node Congress 2022Node Congress 2022
26 min
It's a Jungle Out There: What's Really Going on Inside Your Node_Modules Folder
Top Content
Do you know what’s really going on in your node_modules folder? Software supply chain attacks have exploded over the past 12 months and they’re only accelerating in 2022 and beyond. We’ll dive into examples of recent supply chain attacks and what concrete steps you can take to protect your team from this emerging threat.
You can check the slides for Feross' talk here.
6 min
Charlie Gerard's Career Advice: Be intentional about how you spend your time and effort
Featured Article
When it comes to career, Charlie has one trick: to focus. But that doesn’t mean that you shouldn’t try different things — currently a senior front-end developer at Netlify, she is also a sought-after speaker, mentor, and a machine learning trailblazer of the JavaScript universe. "Experiment with things, but build expertise in a specific area," she advises.

What led you to software engineering?My background is in digital marketing, so I started my career as a project manager in advertising agencies. After a couple of years of doing that, I realized that I wasn't learning and growing as much as I wanted to. I was interested in learning more about building websites, so I quit my job and signed up for an intensive coding boot camp called General Assembly. I absolutely loved it and started my career in tech from there.
 What is the most impactful thing you ever did to boost your career?I think it might be public speaking. Going on stage to share knowledge about things I learned while building my side projects gave me the opportunity to meet a lot of people in the industry, learn a ton from watching other people's talks and, for lack of better words, build a personal brand.
 What would be your three tips for engineers to level up their career?Practice your communication skills. I can't stress enough how important it is to be able to explain things in a way anyone can understand, but also communicate in a way that's inclusive and creates an environment where team members feel safe and welcome to contribute ideas, ask questions, and give feedback. In addition, build some expertise in a specific area. I'm a huge fan of learning and experimenting with lots of technologies but as you grow in your career, there comes a time where you need to pick an area to focus on to build more profound knowledge. This could be in a specific language like JavaScript or Python or in a practice like accessibility or web performance. It doesn't mean you shouldn't keep in touch with anything else that's going on in the industry, but it means that you focus on an area you want to have more expertise in. If you could be the "go-to" person for something, what would you want it to be? 
 And lastly, be intentional about how you spend your time and effort. Saying yes to everything isn't always helpful if it doesn't serve your goals. No matter the job, there are always projects and tasks that will help you reach your goals and some that won't. If you can, try to focus on the tasks that will grow the skills you want to grow or help you get the next job you'd like to have.
 What are you working on right now?Recently I've taken a pretty big break from side projects, but the next one I'd like to work on is a prototype of a tool that would allow hands-free coding using gaze detection. 
 Do you have some rituals that keep you focused and goal-oriented?Usually, when I come up with a side project idea I'm really excited about, that excitement is enough to keep me motivated. That's why I tend to avoid spending time on things I'm not genuinely interested in. Otherwise, breaking down projects into smaller chunks allows me to fit them better in my schedule. I make sure to take enough breaks, so I maintain a certain level of energy and motivation to finish what I have in mind.
 You wrote a book called Practical Machine Learning in JavaScript. What got you so excited about the connection between JavaScript and ML?The release of TensorFlow.js opened up the world of ML to frontend devs, and this is what really got me excited. I had machine learning on my list of things I wanted to learn for a few years, but I didn't start looking into it before because I knew I'd have to learn another language as well, like Python, for example. As soon as I realized it was now available in JS, that removed a big barrier and made it a lot more approachable. Considering that you can use JavaScript to build lots of different applications, including augmented reality, virtual reality, and IoT, and combine them with machine learning as well as some fun web APIs felt super exciting to me.


Where do you see the fields going together in the future, near or far? I'd love to see more AI-powered web applications in the future, especially as machine learning models get smaller and more performant. However, it seems like the adoption of ML in JS is still rather low. Considering the amount of content we post online, there could be great opportunities to build tools that assist you in writing blog posts or that can automatically edit podcasts and videos. There are lots of tasks we do that feel cumbersome that could be made a bit easier with the help of machine learning.
 You are a frequent conference speaker. You have your own blog and even a newsletter. What made you start with content creation?I realized that I love learning new things because I love teaching. I think that if I kept what I know to myself, it would be pretty boring. If I'm excited about something, I want to share the knowledge I gained, and I'd like other people to feel the same excitement I feel. That's definitely what motivated me to start creating content.
 How has content affected your career?I don't track any metrics on my blog or likes and follows on Twitter, so I don't know what created different opportunities. Creating content to share something you built improves the chances of people stumbling upon it and learning more about you and what you like to do, but this is not something that's guaranteed. I think over time, I accumulated enough projects, blog posts, and conference talks that some conferences now invite me, so I don't always apply anymore. I sometimes get invited on podcasts and asked if I want to create video content and things like that. Having a backlog of content helps people better understand who you are and quickly decide if you're the right person for an opportunity.What pieces of your work are you most proud of?It is probably that I've managed to develop a mindset where I set myself hard challenges on my side project, and I'm not scared to fail and push the boundaries of what I think is possible. I don't prefer a particular project, it's more around the creative thinking I've developed over the years that I believe has become a big strength of mine.***Follow Charlie on Twitter
Node Congress 2022Node Congress 2022
34 min
Out of the Box Node.js Diagnostics
In the early years of Node.js, diagnostics and debugging were considerable pain points. Modern versions of Node have improved considerably in these areas. Features like async stack traces, heap snapshots, and CPU profiling no longer require third party modules or modifications to application source code. This talk explores the various diagnostic features that have recently been built into Node.
You can check the slides for Colin's talk here. 
ML conf EU 2020ML conf EU 2020
41 min
TensorFlow.js 101: ML in the Browser and Beyond
Discover how to embrace machine learning in JavaScript using TensorFlow.js in the browser and beyond in this speedy talk. Get inspired through a whole bunch of creative prototypes that push the boundaries of what is possible in the modern web browser (things have come a long way) and then take your own first steps with machine learning in minutes. By the end of the talk everyone will understand how to recognize an object of their choice which could then be used in any creative way you can imagine. Familiarity with JavaScript is assumed, but no background in machine learning is required. Come take your first steps with TensorFlow.js!
JSNation 2023JSNation 2023
22 min
ESM Loaders: Enhancing Module Loading in Node.js
Native ESM support for Node.js was a chance for the Node.js project to release official support for enhancing the module loading experience, to enable use cases such as on the fly transpilation, module stubbing, support for loading modules from HTTP, and monitoring.
While CommonJS has support for all this, it was never officially supported and was done by hacking into the Node.js runtime code. ESM has fixed all this. We will look at the architecture of ESM loading in Node.js, and discuss the loader API that supports enhancing it. We will also look into advanced features such as loader chaining and off thread execution.

Workshops on related topic

Node Congress 2023Node Congress 2023
109 min
Node.js Masterclass
Workshop
Have you ever struggled with designing and structuring your Node.js applications? Building applications that are well organised, testable and extendable is not always easy. It can often turn out to be a lot more complicated than you expect it to be. In this live event Matteo will show you how he builds Node.js applications from scratch. You’ll learn how he approaches application design, and the philosophies that he applies to create modular, maintainable and effective applications.

Level: intermediate
Node Congress 2023Node Congress 2023
63 min
0 to Auth in an Hour Using NodeJS SDK
WorkshopFree
Passwordless authentication may seem complex, but it is simple to add it to any app using the right tool.
We will enhance a full-stack JS application (Node.JS backend + React frontend) to authenticate users with OAuth (social login) and One Time Passwords (email), including:- User authentication - Managing user interactions, returning session / refresh JWTs- Session management and validation - Storing the session for subsequent client requests, validating / refreshing sessions
At the end of the workshop, we will also touch on another approach to code authentication using frontend Descope Flows (drag-and-drop workflows), while keeping only session validation in the backend. With this, we will also show how easy it is to enable biometrics and other passwordless authentication methods.
Table of contents- A quick intro to core authentication concepts- Coding- Why passwordless matters
Prerequisites- IDE for your choice- Node 18 or higher
JSNation 2023JSNation 2023
104 min
Build and Deploy a Backend With Fastify & Platformatic
WorkshopFree
Platformatic allows you to rapidly develop GraphQL and REST APIs with minimal effort. The best part is that it also allows you to unleash the full potential of Node.js and Fastify whenever you need to. You can fully customise a Platformatic application by writing your own additional features and plugins. In the workshop, we’ll cover both our Open Source modules and our Cloud offering:- Platformatic OSS (open-source software) — Tools and libraries for rapidly building robust applications with Node.js (https://oss.platformatic.dev/).- Platformatic Cloud (currently in beta) — Our hosting platform that includes features such as preview apps, built-in metrics and integration with your Git flow (https://platformatic.dev/). 
In this workshop you'll learn how to develop APIs with Fastify and deploy them to the Platformatic Cloud.
JSNation Live 2021JSNation Live 2021
156 min
Building a Hyper Fast Web Server with Deno
WorkshopFree
Deno 1.9 introduced a new web server API that takes advantage of Hyper, a fast and correct HTTP implementation for Rust. Using this API instead of the std/http implementation increases performance and provides support for HTTP2. In this workshop, learn how to create a web server utilizing Hyper under the hood and boost the performance for your web apps.
React Summit 2022React Summit 2022
164 min
GraphQL - From Zero to Hero in 3 hours
Workshop
How to build a fullstack GraphQL application (Postgres + NestJs + React) in the shortest time possible.
All beginnings are hard. Even harder than choosing the technology is often developing a suitable architecture. Especially when it comes to GraphQL.
In this workshop, you will get a variety of best practices that you would normally have to work through over a number of projects - all in just three hours.
If you've always wanted to participate in a hackathon to get something up and running in the shortest amount of time - then take an active part in this workshop, and participate in the thought processes of the trainer.
TestJS Summit 2023TestJS Summit 2023
78 min
Mastering Node.js Test Runner
Workshop
Node.js test runner is modern, fast, and doesn't require additional libraries, but understanding and using it well can be tricky. You will learn how to use Node.js test runner to its full potential. We'll show you how it compares to other tools, how to set it up, and how to run your tests effectively. During the workshop, we'll do exercises to help you get comfortable with filtering, using native assertions, running tests in parallel, using CLI, and more. We'll also talk about working with TypeScript, making custom reports, and code coverage.