Five Ways of Taking Advantage of Verdaccio, Your Private and Proxy Node.js Registry

Rate this content
Bookmark

Verdaccio is an open-source lightweight private proxy registry made in JavaScript with an entirely optional configuration that allows you to publish Node.js private packages and proxy from other remote registries. In this talk, you will learn five ways to take advantage of Verdaccio to improve your workflows and productivity.


You can check the slides for Juan's talk here.

32 min
18 Feb, 2022

AI Generated Video Summary

Verdash is a lightweight and proxy private Node.js registry that allows you to host and publish private packages. It offers five ways to use it, including publishing packages with npm workspaces and handling conflicts. Verdash improves project productivity and continuous integration by caching dependencies and versions. Security considerations are important, and Verdash provides security features like rate limiting. It can be used as a registry for open source projects and offers flexibility and customization options.

1. Introduction to Verdash

Short description:

Hi, I'm Juan Picado. I will show you five ways to use Verdash for a private and proxy Node.js Registry. Verdash is a lightweight and proxy private Node.js registry that allows you to host and publish private packages. Install Verdash globally, run the command, and you have a registry with a user interface to browse private and public packages and dependencies. Let's start with the first way.

Hi, thanks for joining my talk. My name is Juan Picado. I'm here to show you five ways of taking advantage of Verdash to private and proxy Node.js Registry. So, let's begin.

First of all, something about me. My name is Juan Picado. I'm senior frontend engineer at Mobility Dare, it's a brand of Ativinta. I'm based in Berlin. And also I spent part of my time and doing Open Source, mostly maintaining the Verdash project. So, which if you don't know it, you will know more in the next 20 minutes and hopefully you like it.

So, let's get started. So, we are Node.js developers and also JavaScript developers. So, I can imagine you have already published a package and you haven't done this yet. Then you will enjoy this talk because it's very, very straightforward to do it locally.

So, Verdash is a lightweight and proxy private Node.js registry, which entirely optional configuration, which does allow you just to host and publish private Node.js packages. And it's compatible with any package manager. So, this diagram here describes a simple road trip of a request where the private registry is always in the middle between the remote, which can be one or more. And then you have a local cache. And this local cache really benefits. I want to show you how to use it in your project.

So, first of all, you need to install Verdash. And for that, you need to install the global package globally. And that's it. So, just that simple, you don't need to do anything, just getting started. Then the next thing you have to do, is just run command Verdash. Yeah, that's all. So, you have a registry which is running with a user interface to browse, not only private packages, but also dependencies, even if they are public ones. Because Verdash, we don't load any package request via package manager or user interface. To help you to understand I have prepared five basic ways, and how Verdash can be really useful to any JavaScript developers nowadays.

So, let's start with the first one.

2. Publishing a Package with npm Workspaces

Short description:

Which is personal development. We are going to publish a package using npm workspaces. The project structure consists of 5 modules, with one module referencing others. Login to Verdash using the 'login' command and specify the registry URL if needed. Publish the package using 'slack workspaces'. The packages can be viewed on the user interface, along with their dependencies.

Which is personal development. So, we are going to publish a package. For this example, I have an npm workspaces, which is pretty simple and we will see now. And the idea is to publish some packages.

So, first of all let me show you the structure of the project, which is not much. Just 5 modules. And one of them, this is the configuration for npm workspaces, and one of them it has reference to others inside. We will see why I am doing this when we publish and I showed you on the itself.

So the first thing you have to do is run Verdash. And if you want to publish you need to login. For that you can use the command login. And if you want to point to another registry then use the flag registry and the ul of the registry. In this case localhost 4873, which is the default port of Verdash. You login and then if you don't have any user just use anything you want. The password and the email is not important because Verdash does not use it. And then you are login. You can see the server is reacting to any command you are typing. This is because the package manager is big with the registry API.

So now let's publish a package. And if you have used workspaces in npm you can publish several packages at the same time just using these slack workspaces. And this is what we are going to do and just happen. Yes we did it. So we have packages on the registry. So let's actually see how these packages look like on the registry. And this is the user interface. You can switch between dark mode. And here we have the five packages. Awesome. And these are the dependencies I showed you before. So you can just navigate through them and see who has the dependencies of whom.

3. Publishing Packages and Handling Conflicts

Short description:

I just published a package in just a few seconds. If you publish a private package with a version that already exists on the registry, you will get a 409 error. To bypass this, add a suffix to the version. Berdacho merges your local version with the upstream versions. Publishing in an offline environment requires enabling publish offline in the configuration. Remove the proxy property to improve the offline experience.

And that's it. So just like that. I just published a package in just a few seconds. Yes but there is one constraint you must be aware. If you publish a package, a private package, it's which the version doesn't exist on any of the upstreams which also we named it Applinks, like VAC17 for instance, if it already exists on the registry and Berdacho always will look out for this version if it already exists and if it does condition it's true then you will get 409 error. This is quite common if you want to patch a public version, right?

So for bypassing these, you can just add a a suffix on the version. And your version won't be lost once you Berdacho updates new versions because Berdacho merges what you have local with what you have from the upstreams. So let's make a quick demo as well just to show you the benefits and how we can actually solve this with this package react. Let's see if I found the project... Ah, right here. So here we can see this version, it's the current one on npmjs. Let's run Berdacho again. We are already logged in so we don't need to log in again. And let's try to publish this version with the flag registry. Boom! Yes! Publish conflict. So to solve this you can actually change the version manually, but it's much easier if you use a version patch. And in this case I'm using the not github version because I just cloned this project, I don't want to create a local tag in my project. So I just run this version and it will change automatically. I can run it several times and you can see how the version changed according to the previous one. And now let's publish this package. Just publish and there it is, so if we go back to the registry again we can see if I find it, we can see the latest version is the one I just published. If I access to the React version we can navigate through the version so I can see the ones that are coming from outside plus the one I just published. So let's continue with the next slide.

So publishing is easy but if you are working in an offline environment like me in this video working on a plane which is quite common if you are travelling or you live in remote areas or there is no connection at all. If you want to publish packages, and this is the reason because Vertel exists, because you can publish packages everywhere, then you need to enable the publish offline in the configuration. This is not by default because the reasons are because registry always look up for versions in remotes. If there is no connection the publish will fail. Also you can improve the offline experience by removing the proxy property in the configuration file. You can see on the slide. You will see more about this in the next slides.

4. Project Productivity and Continuous Integration

Short description:

Most of the public packages we use are published through private registries, but also call from other registries. Private registries and proxies are key to your development pipeline, preventing downtime during public outages. Verdash caches dependencies and versions, speeding up builds and reducing external requests. Configurable cache intervals and max failures enhance the developer experience.

There is a reason for that. Next is project productivity and continuous integration in private registries. Most of the public packages we use nowadays are being published through the private registry, but also call camp from other registries. All these registries are services that require high availability, but this is not 100% of the time, because public outages happen, and then you must be prepared for it.

In those situations is when the capabilities of the private registries and proxies are fundamental key of your development pipeline. Because having a proxy registry between your pipeline and your public services it will avoid your teams or your builds to stop working while an outage is happening. Verdash will have this capability out of the box, it will cache any dependency and version by demand, so the next time it's requested it will serve the one it's cached. Because in the continuous integration environment the builds will run multiple times in the same pull request when you push changes in your repository, which involves the package manifest for a public package will be downloaded over and over and over.

There is some default cache in Verdash with 2 minutes but you can increase the cache by just telling the registry I don't need a new version every 2 minutes, maybe 10 minutes is fine for me. This is a really good practice because you will speed up your builds because the registry won't need to go for every request outside to ask whether you have new changes. Also if the network is not working fine you can also increase the max failures on the configuration. All these setups are based on a client-side approach with some friendly intervals like 2 weeks, 3 weeks, 30 seconds and so on. This makes it much easier for the developers to understand what actually are the values you have set up.

5. Security Considerations for Registries

Short description:

Security is a crucial topic that needs to be addressed from both the client-side and server-side. A misconfiguration on registries can lead to private packages being replaced by malicious ones on public registries. It is important to properly configure the registry and isolate private packages using scopes. If using Verdacho, removing the proxy in the configuration is recommended for private packages.

And the third topic is security because security is a topic that must be tackled from 2 angles. First, the client-side because you need to use the right configurations in the Package Manager. You can check more about this on the virtualwebsite. I also have a link on the 2 slides forward. But also from the server-side because it raises itself and I will elaborate a bit more now because there's a reason. And if you don't remember what happened 1 year ago, let me quickly recap. A security researcher discovered that a misconfiguration on registries might help intended to be private packages being replaced by malicious packages, which are published on public registries with the same name. This might compromise security. This story is too long, but here I leave a nice article where you can read more about it. That's just a reminder that a registry is a great advantage and brings you an extra layer of security. But also it's a big responsibility because it must be properly configured. There is two actual advices here. First, isolate your private packages by using scopes. Usually is the name of your company, of your project, or organization. And also try to register this name in public registries. Because this is a way that you can have full control. In case you haven't configured properly the registry locally, then at least you own the scope outside. The second reason is, especially if you're using Verdacho, please remove the proxy in the configuration to avoid the registry to fetch external sources for updates. This is highly recommendable for scope packages. If you are intend to be private, especially if you're using Verdacho.

6. Verdacho Security and Package Testing

Short description:

The latest release of Verdacho includes a new security feature: rate limiting. This feature can be applied to web endpoints and security endpoints like login and change password. End-to-end testing is crucial to ensure the integrity of your packages. Misconfigurations are common pitfalls when shipping a package, so testing and using a registry is recommended. Let's look at an example of fixing a broken package in a GitHub project using Verdacho and GitHub Actions.

So the latest releases of Verdacho, it includes a new security feature, which is rate limit. At this point, you can add rate limit to two areas, which are the web endpoints, the security endpoints, which are the login and change password. With that, you're going to use the possibility of DNL of service attack, which these are key in points. So by default are actually really set up to really low number of requests, but you can change this. So I totally recommend to use this feature if you haven't noticed it already exists.

So, end-to-end testing, which is my favorite one, which the integrity of your packages is when you are publishing a package and the structure of functionality of these models are not being affected when it's consumed by users. There are so many ways to break a package, the most common one is misconfiguration, because it does not matter how well tested your model is. There are some main steps when you are configuring to ship a package that depends on the developers, and it could be you forgot to add main field if you are trying to ship a common-JS package, or maybe you forgot the module if you are trying to use a JSON module, or maybe you forgot the right patterns on the file property in the package. Also, you might misconfigure the .npm.ignore file in the root file. There are so many things involved on shipping a package, and any mistake counts and can be cached only in publishing. For that, you need a registry. You cannot publish snapshots all the time to the public one, especially if I private packages, so I recommend to test publishing your packages on every pull request, just like a storybook is doing on this slide, for example.

Let's see an example, I can break a package that is intended to be shipped as a command-line interface. Here I have a demo, if you want to look it up later, it just prints hello world on the console, and it uses github actions, docker and node js to run small end-to-end tests. So, as you observe, this is a view of my github project, which my tests are broken on master. So, let's see quickly what's the reason, and let's fix it in a moment. So, this is my project, and if you see, there is something wrong on the config, on the package. So, let's see for a second what might be, and it seems it's thebin file. So, this file does not exist on the root, it doesn't exist. So, what we can do to fix it? Well, just using the right name, right? And, let's actually commit this file because if we... and this should actually fix the problem. So, let's commit this. No, I don't want break master, I want to fix master. Yeah, let's push these changes and then while GitHub runs the action, I will show you how I'm actually doing this test. So, if you're not familiar with GitHub Actions, it's really simple, I would totally recommend to use them. Here we have service. Service is a property that allows you to use docker images within your action. In this case I'm using the Verdacio docker image, which is the official one. I'm exposing two ports, I mean one port, which is the same. For the S73, with this you can use the registry in all the steps below.

7. Publishing and Testing Packages

Short description:

First, log in and publish the package. Then, check if the package is available and install it globally. Run the test, and if the outcome is fine, everything is good. Otherwise, process x with one. The GitHub action shows that the master is green again.

So, first I'm just logging because yes, you need to log in the first time. Then I publish the package. This is what any user will do. Then, I just check whether the package is there, which is optional. Then I just install globally the package. When you're installing globally an OGS package using VeeN, this command is gonna be available on the global context. And then I run the test. In the test I don't have anything special, I'm just using a chilled process, which uses SpanSync. Running the command, I'm just supposed to be available on the global context. This will be a first breakpoint if there is no command. This test is gonna fail. Then it's gonna grab the outcome. And then we compare the outcome, if it's fine, everything's fine. Otherwise, I process x with one, which means the GitHub will fail. Let's go to the GitHub action. Yeah, two minutes ago. Fixed master is totally working fine. If we see the action inside, we can see that this actually passed correctly. Perfect. And the master is green again. Super nice. So let's go back to the slides.

8. Using Verdacho as a Registry

Short description:

There are many examples of open source projects that can be used as a learning resource. Angular CLI is one such example, which uses Chill for running Verdacho. Hosting a registry may seem daunting, but Verdacho provides a server and user interface in the same package, making it a convenient and cost-effective solution. Verdacho is flexible and can be customized to meet different requirements, such as using different storage options or implementing custom authentication providers. Overall, Verdacho is an open source registry designed for developers and contributors, offering privacy, flexibility, and low cost.

There's many examples on open source, and the best way to learn is a good example is reading open source. For instance, you can use Angular CLI, which also uses Chill for running Verdacho. In this case, they are using Verdacho programmatically to run the registry, and this is also a nice way to do it. So if you want more examples, just ping me on the chat and I can give you more, because I have plenty of more to share.

Um, because having your own registry might be intimidating. How hard is hosting a registry? How much cost? How much effort? All these questions are common, and a registry needs two fundamental pieces, the server and the user interface, and Verdacho has both in the same package. Also, you must ask yourself, what are you expecting from a registry? Because Verdacho fits better if you're looking for privacy, flexibility, customization, all offline environments, and overall low cost. Because this is what we are doing in Verdacho organization, we are dogfooding our own registry and our builds.

We have been running our own registry for three years, which around costs $60 per year. And we have thousands of builds every month, including all the dependables on all these people contributing, which is plenty of requests to the registry every day. And we haven't had the need to scale so far because it has been working really great with the default configuration. But this is our own personal setup. You might have different requirements, and Verdacho is flexible. You might need a different storage. For instance, you might use S3 back in Amazon Web Services or your own Cloud like Minio. You might need your own authentication provider or maybe change the interface completely. Verdacho also allows middleware because it's express at this point. This is one example. I'm using MPM Audit. We are applying using plugins giving support to this command. So you might want to extend your own registry. But yes, this might include also personal development, creating your own plugins.

So yeah, that's all for my side. Thanks so much for listening my talk. Verdacho is an open source registry made for developers and made for contributors, which make this project a great alternative. And so in name of the all contributors, thanks for watching my talk. And hopefully you like it and you learn something today. Please don't hesitate to ask more questions or contribute to open source. So thank you so much. Bye-bye.

QnA

Verdacho Pool Results and Contribution

Short description:

Taking a look at the pool results, it matches with my expectations. Hosting a registry is the most common use case, but there are other interesting topics like security and continuous integration. The best way to contribute to Verdacho is by having the willingness to learn. There are various ways to contribute, such as fixing types, improving code, or helping with translations.

Taking a look at the pool results, what do you think? So one third of all the attendees, well, they care about hosting a registry, but there's also a lot of interesting answers like security. What do you think about that? Actually, I looked at the results, it matches with my expectations. Usually, yeah, the name of the project description indicates, yeah, you can host a registry. This is the most common use case. But it has several ones, like I mentioned in the book. And Antoine testing is a trend, which is getting popular. You saw in the presentation, I mentioned a few projects, but I see many, many projects, which they have monolibos, which they care about what they're shipping. Antoine testing is something important. Security, yeah. Last year was an important year for registries and security. And the project was fork for another one, which it was more intended to be for personal development. This is maybe the less known. You can use a registry in your computer. And what I'm surprised is the continuous integration, because like I mentioned, one of the slides usually is not used. We rely a lot on services and this is dangerous. So yeah, that's pretty cool. Thank you.

So well, we have a few questions, if you don't mind. First of all, this is my question also, and I believe it's going on in the minds of many people. What is the best way to contribute to Vodaccio? What's the best way to get involved? Well, just the willing to learn. That's the most important one. This is how I started in Vodaccio. Yes, I want to learn Node.js and on the repo, we have several ways you can help. First, if you're just a beginner, you can start fixing types we are migrating. We have our roadmap on the repository, which is peanut, and you can easily find it on the issues and discussions. And there is a description what you can do. Either you are a Node.js developer purely and you want to learn something or improve the code, you have the steps how to do it. If you like more of the front-end, we have a UI, which is based on React, and Material UI, which also needs some help there. And yeah, you can have good features. And if you are not a coder, maybe you want to help with translations.

Verdacio Features and Comparison

Short description:

You can help translate the software to multiple languages. Verdacio can be used to proxy multiple registries, allowing you to fetch packages from different sources. You can define rules to find packages in different registries, creating a powerful combination. Verdacio differentiates itself from other projects like Entropec, which has seen limited development.

Also, you can help translate the software to several languages. We have like 15 languages already on the UI and also documentation. And yeah, also you can go to the chat and help others to use Bellagio. That's another way. Thank you very much. That's very cool.

Another question is, can Verdacio be used to proxy not just one, but multiple registries? Yes. When you are using Verdacio the first time and you are installing packages through it, yeah, the packages are not arriving magically. They are coming from the public registry, but you can use several ones. I mean, imagine you have another registry in your company and you want to fetch the packages from there and the public from the public one. So you can set up one or more. There is no limit. I mean, I think the limit is the sky or in this case your hardware. And either is a private, you can use credentials to access to the, in your corporation you can set up credentials and access to several registries. There is one of the features which is not very known. It's like you can define a specific package, for instance, React. Let's imagine. And you can ask for React in your private registry. And you can define a rule, like, okay, if it does not exist on the private one, then I will find it on the other one. And you can create this combination, which is really nice. And, yeah, so it's totally, totally possible. That's really nice.

Well, another question that I had is that I'm sure others might be familiar with the other initiatives to create non-npm repositories. An example would be Entropec, which people might be aware of. How do you think Vodaccio compares or maybe differentiates itself from those projects? Which one? The Entropec you mentioned? Yeah, Entropec. Or is there maybe some other projects that you're aware of that you think are different? There is not many in open source, unfortunately. I would like to see more. But the most of this, Entropec was an idea which is still open, but it seems there is not much development. But it was an idea to create a registry, distributed registry. Yeah, so unfortunately there is not much development there, but the idea was interesting.

Private Registries and Offline Learning

Short description:

There are companies that offer private package registries as services, but they require payment and rely on internet connectivity. However, Verdacho allows you to install it on your computer, providing an offline learning experience.

And aside of that, there is only services. There is some companies which are creating registries, possibilities to private packages. But yeah, I mean you need to pay for them. Some they have some free trial, which you can use them. But the main difference is like, they still are all in services, right? So you need internet and you need basically internet and I can tell you, usually people approach me to ask me like, oh, I live in this area or I live in this country, which is not allowed to use the service or whatever. And my unique way to learn Node.js is because I can actually install on my computer and just use it. This is interesting for me.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Node Congress 2022Node Congress 2022
26 min
It's a Jungle Out There: What's Really Going on Inside Your Node_Modules Folder
Do you know what’s really going on in your node_modules folder? Software supply chain attacks have exploded over the past 12 months and they’re only accelerating in 2022 and beyond. We’ll dive into examples of recent supply chain attacks and what concrete steps you can take to protect your team from this emerging threat.
You can check the slides for Feross' talk here.
Node Congress 2022Node Congress 2022
34 min
Out of the Box Node.js Diagnostics
In the early years of Node.js, diagnostics and debugging were considerable pain points. Modern versions of Node have improved considerably in these areas. Features like async stack traces, heap snapshots, and CPU profiling no longer require third party modules or modifications to application source code. This talk explores the various diagnostic features that have recently been built into Node.
You can check the slides for Colin's talk here. 
JSNation 2023JSNation 2023
22 min
ESM Loaders: Enhancing Module Loading in Node.js
Native ESM support for Node.js was a chance for the Node.js project to release official support for enhancing the module loading experience, to enable use cases such as on the fly transpilation, module stubbing, support for loading modules from HTTP, and monitoring.
While CommonJS has support for all this, it was never officially supported and was done by hacking into the Node.js runtime code. ESM has fixed all this. We will look at the architecture of ESM loading in Node.js, and discuss the loader API that supports enhancing it. We will also look into advanced features such as loader chaining and off thread execution.
JSNation Live 2021JSNation Live 2021
19 min
Multithreaded Logging with Pino
Almost every developer thinks that adding one more log line would not decrease the performance of their server... until logging becomes the biggest bottleneck for their systems! We created one of the fastest JSON loggers for Node.js: pino. One of our key decisions was to remove all "transport" to another process (or infrastructure): it reduced both CPU and memory consumption, removing any bottleneck from logging. However, this created friction and lowered the developer experience of using Pino and in-process transports is the most asked feature our user.In the upcoming version 7, we will solve this problem and increase throughput at the same time: we are introducing pino.transport() to start a worker thread that you can use to transfer your logs safely to other destinations, without sacrificing neither performance nor the developer experience.

Workshops on related topic

Node Congress 2023Node Congress 2023
109 min
Node.js Masterclass
Workshop
Have you ever struggled with designing and structuring your Node.js applications? Building applications that are well organised, testable and extendable is not always easy. It can often turn out to be a lot more complicated than you expect it to be. In this live event Matteo will show you how he builds Node.js applications from scratch. You’ll learn how he approaches application design, and the philosophies that he applies to create modular, maintainable and effective applications.

Level: intermediate
Node Congress 2023Node Congress 2023
63 min
0 to Auth in an Hour Using NodeJS SDK
WorkshopFree
Passwordless authentication may seem complex, but it is simple to add it to any app using the right tool.
We will enhance a full-stack JS application (Node.JS backend + React frontend) to authenticate users with OAuth (social login) and One Time Passwords (email), including:- User authentication - Managing user interactions, returning session / refresh JWTs- Session management and validation - Storing the session for subsequent client requests, validating / refreshing sessions
At the end of the workshop, we will also touch on another approach to code authentication using frontend Descope Flows (drag-and-drop workflows), while keeping only session validation in the backend. With this, we will also show how easy it is to enable biometrics and other passwordless authentication methods.
Table of contents- A quick intro to core authentication concepts- Coding- Why passwordless matters
Prerequisites- IDE for your choice- Node 18 or higher
JSNation 2023JSNation 2023
104 min
Build and Deploy a Backend With Fastify & Platformatic
WorkshopFree
Platformatic allows you to rapidly develop GraphQL and REST APIs with minimal effort. The best part is that it also allows you to unleash the full potential of Node.js and Fastify whenever you need to. You can fully customise a Platformatic application by writing your own additional features and plugins. In the workshop, we’ll cover both our Open Source modules and our Cloud offering:- Platformatic OSS (open-source software) — Tools and libraries for rapidly building robust applications with Node.js (https://oss.platformatic.dev/).- Platformatic Cloud (currently in beta) — Our hosting platform that includes features such as preview apps, built-in metrics and integration with your Git flow (https://platformatic.dev/). 
In this workshop you'll learn how to develop APIs with Fastify and deploy them to the Platformatic Cloud.
JSNation Live 2021JSNation Live 2021
156 min
Building a Hyper Fast Web Server with Deno
WorkshopFree
Deno 1.9 introduced a new web server API that takes advantage of Hyper, a fast and correct HTTP implementation for Rust. Using this API instead of the std/http implementation increases performance and provides support for HTTP2. In this workshop, learn how to create a web server utilizing Hyper under the hood and boost the performance for your web apps.
React Summit 2022React Summit 2022
164 min
GraphQL - From Zero to Hero in 3 hours
Workshop
How to build a fullstack GraphQL application (Postgres + NestJs + React) in the shortest time possible.
All beginnings are hard. Even harder than choosing the technology is often developing a suitable architecture. Especially when it comes to GraphQL.
In this workshop, you will get a variety of best practices that you would normally have to work through over a number of projects - all in just three hours.
If you've always wanted to participate in a hackathon to get something up and running in the shortest amount of time - then take an active part in this workshop, and participate in the thought processes of the trainer.
TestJS Summit 2023TestJS Summit 2023
78 min
Mastering Node.js Test Runner
Workshop
Node.js test runner is modern, fast, and doesn't require additional libraries, but understanding and using it well can be tricky. You will learn how to use Node.js test runner to its full potential. We'll show you how it compares to other tools, how to set it up, and how to run your tests effectively. During the workshop, we'll do exercises to help you get comfortable with filtering, using native assertions, running tests in parallel, using CLI, and more. We'll also talk about working with TypeScript, making custom reports, and code coverage.