A journey of the thousand binaries

Rate this content

In this workshop we will explore the advantages of having a robust, mature universal artifact repository manager at the heart of the software development cycle. We will explore the most important concerns when developing rich applications and adapting to a fast pacing market. In the past years big companies have benefited from techniques like AB testing to improve their products, increase traffic, improve UX experience and offer new functionality. This is only possible if there is a solid devops infrastructure in place with tools that provide among others, control, security, versioning and good annotation support. It’s not only about having the right tools but knowing how to use them to its full potential.

67 min
29 Mar, 2022


Sign in or register to post your comment.

Video Summary and Transcription

The Workshop covers the importance of dependencies in software development, the different types of dependencies, challenges and security issues with dependencies, the role of repositories in dependency management, and the process of resolving dependencies and publishing. It also discusses the configuration of repositories and authentication, and the creation and configuration of different types of repositories. The Workshop emphasizes the need for understanding dependencies, ensuring security, and using tools like JFocusray and OWAS dependency check for analysis and assessment.

Available in Español

1. Introduction to Dependencies

Short description:

This part is about the importance of dependencies in software development. It emphasizes the need to reuse software written by others and discusses the different types of dependencies such as frameworks, libraries, packages, modules, and resources. It also highlights the fact that dependencies are not all the same and can vary in terms of functionality and complexity. The text encourages the audience to start provisioning their free tier and provides a link for them to do so. Overall, the part sets the stage for the workshop and introduces the concept of dependencies in software development.

Hi, hello. I'm super happy that you are here. One of the most important links that you will have is that one that is showing right now in my screen. So, if you follow there, you can get a free instance. This is important because to follow all the examples and to do exactly the same what I'm going to do at the demo so and provisioning instances usually takes a little bit of time. So, it's important that you start doing it at this moment so we reach the hands on part of this workshop, we will have our instances running.

And, well, I'm here in Switzerland, in Basel, Switzerland. I'm super excited and happy that you're here with me. So, let's start with the session or the workshop. And I will post the link and you will have it in some of the slides, so if you join later on or you haven't done it, there is still a chance. So, this is me. I'm from Mexico. I live in Switzerland, as I work for a company so that's exactly where you're going with that link. Again, this session and this workshop is all about dependencies. And if you're starting to provision your free tier, this is what you are looking at. And you go and start for free. You can choose whatever name there. And as I said, it takes time. So, that's why I encourage you to do it right now. And while you do that and everything is provisioned, let's start with the real interesting topics.

So, today's presentation and workshop is about a key part of our software development process. And it is dependencies. We don't need to reinvent the wheel every time we want to achieve a new level of functionality or deliver software faster. We want and we do reuse software written by others every day. Software dependencies. But by doing that, sometimes, we get into trouble. So, dependencies are not only used while we are developing. They are used for runtime or testing. And you know what? Dependencies are not all the same. So, I'm going to give you two statements and you will tell me if they are true or false. Actually, I will answer that for you. But you get the idea. So, dependencies are collections containing high quality tested code that provides functionality that requires significant expertise to develop. That's true. Some of our dependencies are really highly functional. They are really well tested. That's the case of our frameworks. So, it's true. Dependency managers like NPM have made possible that almost trivial functionality can be packaged and published. That is true. Publishing to NPM registry is dead trivial. And these are both sides of the spectrum. From one side we have libraries or pieces of code that require a lot or require a lot of work, have a lot of tests and provide a lot of functionality and maybe they are opinionated. On the other side we have very small pieces of code. So, we have to see what are the different types of dependencies that we have. Frameworks, libraries, packages, modules, and resources. So, this is typical what we refer as our dependencies. So, what is our resource? A resource is a collection of files, for example, templates, media, audio, video or images, plain text or even blobs that need to be included inside our applications to execute or display correctly. This doesn't need to be binaries. This is something that we depend and is needed.

2. Types of Dependencies

Short description:

This part explains the different types of dependencies in software development, including modules, packages, libraries, and frameworks. It emphasizes the importance of understanding the purpose and functionality of each type of dependency in order to make informed decisions. It also highlights the need to consider the degree of need and the impact of dependencies on updates, migration costs, and clean-up efforts. The part concludes by reminding the audience to start provisioning their environment and highlighting the significance of dependencies in software development.

And a module. This is a set of functions or methods that provides self-contained functionality. A module usually has an interface that explicitly and abstractly specifies both the functionality. And it provides an interface and usually one implementation. So, we can see it as a black box.

A package is a collection of modules that hold, in general, the same functional purpose. Usually a directory. And this is only for Python and JavaScript. We have a different structure for a package, but the concept still remains. Usually a directory contains a file that describes metadata about the package. And usually they're aggregated based on the functionality. So, it's easy to abstract or omit some packages, entire packages.

And a library. A library is a collection of related functionality defined in several packages. It's essentially a set of functions that you can call. Each call does some work and returns control to the client or framework or application. And finally, frameworks. A framework embodies some abstract defined with more behavior built in. And in order to use a framework, you usually, it's the other way around. We include our code inside some places in the framework and he will call us. So, for example, this is one of the most interesting difference between a framework and a library. And they usually are more opinionated. So, frameworks or platforms usually are bigger, are more opinionated. The integration between the different functional components is more connected. And they usually have a long list of versions or at least a road map. It's very mature. Because it usually requires a larger group of developers to build its functionality. Usually there is a license and, of course, there is a set of tests.

And in the JavaScript world, this is pretty apparent. Because, for example, Angular is defined as a platform. While React, they said that it's just a library or a group of libraries to build user interfaces. So, they are very clear about what is the purpose of their libraries or frameworks. And on the order side of the spectrum, I told you before, you have frameworks and you have tiny, tiny, tiny packages. And this is a collection of micro NPM packages, like sometimes one liners.

So, there are different types of dependencies. And having a clear mind or a clear idea which type of dependencies we have in our projects make us realize and also make us think about which ones are the crucial, what are the important ones. What are only cosmetics? Others are easily exchanged. And also something that we have to keep in mind is our degree of need. Because even if it's a large library, it's not really needed. And these two factors are really important to help us decide the cadence of update. Migration costs, our clean up efforts. Because that has to happen in every single application. So, for example, we have a dependency on five major libraries. Or five major one platform usually. And five major libraries. When do we update? How do we test? And what is our own roadmap? How dependent are we on code that is not so great? So, this is the question that we need to ask every single time that we decide to add a new dependency in our projects. Or we're trying to think about the future, or we have a problem with a specific dependency. So, making dependencies important, our level of dependency, our level, or the type of dependency is really important.

So, before we continue with the bad news, I will remind you again of the link to start provisioning your environment, so you can work with me alongside me during the demo part. So, we have dependencies, all types of dependencies, and we depend on them in different ways.

3. Challenges with Dependencies

Short description:

When adding a dependency, we outsource the development and maintenance work to someone else. This exposes our code to the failures and flaws in the dependencies, regardless of whether they are open source or closed source. Trusting open source libraries is a challenge, as there is no contractual guarantee or company behind them. However, reputation can be a factor in deciding whether a library is trusted. Open source has its benefits, and I am personally involved and passionate about contributing to it.

So, what could possibly go wrong? When we are adding a dependency, we are also outsourcing the work of developing that code, designing, writing, testing, debugging, and maintaining to someone else. Often, they're a non-programmer. So, using that specific code exposes whatever we're doing to all failures and flaws in the dependencies. And this is not really related to if it's open source or if it's closed source. Actually, there are a bunch of articles that show that there is no real preference of where do you find the most bugs. I mean, I can show you if you're interested in those. I can send you all the references. But again, with open source, open source changes a lot of things. In the past, before it came to our lives, we developers trusted others to write this or that they depend on. I'm happily younger than that. So I started with open source already. So we usually or they usually buy the software. So for example, it was the compilers, or the drivers for a specific hardware. So there was this contractual idea that if there was something wrong, there was a company or a product behind them. And with open source it's not like that. We do have licenses that says as is. And how do we decide that this is a trusted library or not? Sometimes it's based on reputation. Don't get me wrong, I'm not saying open source is bad. I'm actually one person that is super involved in open source. And I love open source. I'm always looking forward to give back, to contribute in a smart way or another.

4. Security and Trust Issues with Dependencies

Short description:

There have been several security and trust issues in the JavaScript and NPM ecosystem in recent years. Examples include unpublished packages causing disruption, ownership changes breaking Linux systems, compromised credentials leading to stolen data, and malicious packages causing outages and stealing tokens. To address these issues, tools like JFocusray, OWAS dependency check, and Scorecards can help analyze and assess the security of dependencies. Additionally, an article in ACMQ suggests asking questions about documentation, API design, code quality, testing, bug fixing, and maintenance to evaluate the state of dependencies.

And so, what has happened in the in like what are having the bad news in the JavaScript world or in the NPM environment or ecosystem in the last years? Well, we all remember in March 2016, a package called LeftBatch which was very popular with a lot of JavaScript libraries. After a naming dispute, it was unpublished. And it caused so there's so much disruption that NPM had to republish again three hours later. And they now change or after this event, they change their unpublishing policies.

The next one that made the news was in February 2018, there was an issue with the NPM version 570. And it turns out that in Linux systems, it changed the ownership of the file system. So, they actually broke the machine, the Linux machines. In July 2018, the NPM credentials of one of the maintainers of ESLint scope were compromised. So, there was a release on version 372 and where in this particular release, what happens is the NPM credentials of the machine running ESLint scope were stolen and uploaded. In November 2018, it was discovered that there was a new version 336 of event stream. This particular package included a flat map strip that contained encrypted payload that stole bitcoins from some applications. And this is already the two more recent ones and I had to add a new one but we will see that slide in April 2020. A small package called ISPROMIS resulted in an outage in several list applications. In January 2022, and probably this you heard about it, the maintainer of colors pushed changes printing garbage text in an infinite loop. And this was what really happened to Aron Schwartz. And most recently, this was, let me see, this was at the beginning of each year, there was some malicious NPM packages that stole the scoring tokens. And the most recent one, this was last week, and the developer behind a huge popular NPM package node, IPC, released a subatomic version of the library. Actually, this particular version was not so downloaded until the Rhea evangelist added the modulus to the dependency. Well, actually the dependency to node, it was the malicious package was not node IPC. So, it was a dependency that was added by Rhea evangelist to node PC that make it for example, there was Vue JS dependent on it. So, that caused the problem. So, there are some issues here. There are some security and trust issues in the community. There are some security and trust issues with the code that we consume. And there are tools to help us. And of course, one of it and we are going to test it right now, well, in a few minutes, is JFocusray, an application security SEA tool attached to Artifactory, and it does full binary analysis. And not only binary analysis, that also checks deeply into all the layers of all the dependencies, for example, if you have Docker images, zip files or packages. The other one that I recommend you is to the OWAS dependency check. This is something maintained by this foundation, and it will tell you which of your dependencies have any issue. The third one, but this is most if you are an open source developer, or maintainer or project lead. Scorecards is a project that runs different heuristics associated with software security and assigns a score. So, from zero to ten. And it provides a report, this report is sent to the Linux Foundation and they get more information about what is the state of open source and the security in open source projects. So, that helps them. And it also helps the potential users of consumers of the different libraries. So, totally suggest you to use those. And finally, this is more for for starting the discussions with your managers or take it upstream. There is an article in the ACMQ. And it's about surviving software dependencies. And he is one of the things that he talks about is that the expected cost is all of having a failure in your application. It depends on all the sums of all the cost but the probability of something happened in or something happening as a risk. So, he says, to prevent that or to reduce this, we should make some questions about how or what is the status or the state of our dependencies. Actually, some of these questions actually inspire the scorecards project, or they at least intersect. So, first of all, you have to check is there a documentation? That is usually a good sign, even though some developers hate to do documentation. Others love it, but documentation is important. Are the APIs well designed? Is there a clear design, a consistent design? Do we call the parameter the same way in different versions? Well, not in this version, because there can be a breakage, but at least in similar APIs in the same version, can you see that there is a clear design in the set of APIs or the documentation? Code quality, we have to double check on that. I'm not saying that we run a static code analysis in the source code of all our dependencies that we are introducing, because clearly we have direct dependencies and transient dependencies, but again, in the ones that you depend a lot, this is important, because it's going to hurt more. Well, we can go into that discussion. So the code quality, is the code well written? Is this something that you would like to debug in case something goes wrong? Testing, are there tests for this particular library? Can you run them? Do they pass? Do they specify the basic functionality? Bug fixing, do they track bugs, or the project is like, oops, I did it again, and you'll figure it out? How many bug reports are open? How often do they close it? Do they answer? Maintenance, how often is it maintained? How frequent? How many people are working on the package? If it is used by a large community, chances are if the project leads are having to step down, somebody else will take the lead. So that's good.

5. Security and Dependency Management

Short description:

This part discusses the importance of security in dependencies. It highlights the need for sanitizing inputs, publishing credentials responsibly, and ensuring compliance with licenses. The part also emphasizes the relationship between direct and transient dependencies and the problems that can arise from outdated versions. It mentions the role of repositories in solving dependency issues and encourages the audience to provision their instance of Artifactory. The part concludes by explaining the steps involved in cloning an MPM project and running MPM install, highlighting the large number of dependencies that are added in the process.

This is also depending in the same direction. Security. Do they sanitize their inputs if they are using weird inputs? So there is some security um, gadgets that can help you. Do they publish like credentials? Are they so naïve that do that in their own project? So probably if they are doing that, they are not thinking about security in the first place. Do they have a license? Can you import it in your in your code without any problem later on with compliances? Believe me, you don't want to have discussions with compliance.

So again, as I said, we have direct dependencies and transient dependencies. If you're a small, really small dependency, it's going to bring you 400 more dependencies. I mean a small dependency means like you don't need it too much, but it's going to bring you into your project a lot of transient dependencies with updated versions. The outdated versions, then you still you need to start thinking about what is a problem. And dependencies doesn't exist in the vacuum. They exist in repositories. So most of the problems that we have about dependencies and with dependencies can be solved at the origin where we store all kinds. This is why in the first place I came to work for JFrog, because before this I was a consultant, a senior developer. Mostly I'm a full stack developer. But as you saw in my introduction slide, I'm usually a server side developer. So for us in the Java world, it's typical that we have a lot of effort in creating our repositories close to our development environment. Probably it's in all languages. But I say it like we have this particular idea. So again, if you have a provision, your instance of artifactory, I urge you to do it. And because we will start the demo now. But before I start the demo, I'm going to show it in a screenshot. And I'm going to go through what are the steps we're going to do. And finally, we're going to do it together. So this is the moment where I tell you what are we going to do and what are the benefits of what we are doing. So as I said, I'm an open source proponent. First and foremost. So instead of giving you a repository of the hello world application, what I would like to do is, yes, I can send it to you. This is the one, I mean, there is yes. Don't worry. There was a question in the chat. So this, one of my first and foremost ideas is that as developers, we spent almost the same amount of time reading our own code than reading older developers' code, so being able to go deep in older's code should be something that we show a lot and we do a lot because this is part of reality. This is part of our daily lives. So what we are going to do is to clone, oh my goodness, this. To clone the repository from this particular MPM project, R3. This is one of the micro MPM packages that I showed you in the video at the beginning. So it should be very small and very trivial. So that is the point. So what we're going to do is we're going to clone it. And we are going to check, after cloning, you can see that there is a package. So actually, it's a really, really small project. And the first thing I'm going to do is run the MPM install. And in the MPM install, out of the box, nothing has changed. We can see that the dependencies are resolved to the registry MPM. Yes. And it takes five seconds to install and add all of the 172 packages that this particular tiny project requires. I'm not criticizing. I know this is like normal. This is nothing to be like surprised by. Don't worry. And what we're going to do is going we're going to delete it.

6. Setting Up Repository and Creating User

Short description:

In this part, we delete our node modules and run local with variables. We add the npm configuration file and the repository. We explain the process of creating the repository and the importance of completing the start for free page. We emphasize the need to verify the account and choose a repository name and cloud provider. We highlight the time it takes to set up and the questions asked during the process. Finally, we create a new user and an NPM repository with a specific type.

And I will tell you why. We're going to delete again our node modules. And later on, we will run again local with variables. And why I'm doing this particular tree code the three pieces of code. The first one is because I'm going to add my npm configuration file. Because I'm going to add my repository. You were going to say like what repository, Ichel? Well, the one that we just created with the link. So, let's see that.

Again, the link. This is super important. And at this time, I'm going to explain what should have happened already. So, we should have gotten into this particular page. Like start for free. We enter our name, our last name, email, and password. You need to verify your account if you want to keep it. So, I encourage you to do that. So, you can choose your own name of your repository, and you can choose your own cloud provider. In this case, I'm going to select AWS, I'm going to call it something like DevOps JS something. And I'm going to use the central location for my cluster or my cloud provider. And as I said, this was the reason why I was so adamant on please do it, please do it! It is because it takes time. So, it has to show you what is the name and what did you choose about? And after a while, it will tell you yes, perfect, you are there. And it will ask you some questions, don't worry. And it will ask you why are you using it or how do you want to use Artifactory. It is super powerful and polyphonic. Sometimes it will show you different things on your dashboard and how you answer this. But don't worry, if you didn't answer this, or you selected everything or nothing, actually nothing is possible, but everything, it doesn't matter. I can show you when we are doing it together what was the issue. And then this happens. You have a new repository. You have your new dashboard that is, it says on one side, and the applications.

And the first step I always, will always recommend you is to create a new user. And in this case, the user I like to create all the time is my admin user. So, you go into the second tab and then we have a different submenu and then you go to the users. So, don't worry. We will do it step by step later on when I do it really hands on. This is just for you to have, like, a clear idea of what we're doing. I don't expect you to do it right now. Just sit down still for the movie. So, create a new user and we're going to create a new NPM repository. See, I already changed to my admin user. And I return to the first tab, where it's my repositories. So, I'm hitting the quick setup button. And that happens. I have all the different types that my repositories can have. I mean, as I said, Artifactory supports several and also supports the generic. What is the advantage of using a particular repository type? The advantage is that we can index based on different data. I mean, not all our binaries or files have the same properties. And sometimes, indexing them by the right or the most common or the required ones makes it faster. So, if you I encourage you to use a specific type instead of the generic. Then we're going to create a new repository.

7. Artifactory Repositories and npm Configuration

Short description:

This part explains the concept of local, remote, and virtual repositories in Artifactory. It highlights the benefits of virtual repositories, which allow for a mismatch of different repositories and easy segmentation and control. The part also discusses how Artifactory efficiently stores data by linking shared dependencies instead of duplicating them. It emphasizes the importance of virtual repositories for efficient resource management and easy promotion of binaries. The part concludes by explaining the process of configuring an npm repository and the steps involved in using it for dependency management.

And here you will see that we have three tabs. And I will explain those three. So, you can have local, remote, and virtual repositories. And your local repositories is where your bills, your binaries, your files, your libraries, the ones that you created. Everything about you is there. And see that's we will see a tab with the repositories and it says local. And it will always tell you in the information button what is happening.

The remote one. This is for your third party libraries, libraries from teams and locations. This is where it's like a proxy and it will be a cache of those dependencies. And we will see what do I mean by that. And, again, the second tab, it says what and it actually shows you what are your repositories by each type. And you would have noticed that there is a lot that says default. I'm going to explain that. After I said, what are the virtual repos? And this is actually the most interesting feature for me when I was my previous to my life at JFrog. Because in a virtual repository you can have a mismatch of different repositories. And instead of having a list of all the different repositories or reference your particular dependencies to with a long list this was our solution. So let me tell you about why I was so with this. As a Java developer in my previous life as a consultant I used to go to companies like banks or actually yes living in Switzerland you can imagine who were my clients like banks pharmaceuticals and of course banks and pharmaceuticals are really interested in security. So we had all our development environments and testing and every single step of our building process they have their own repositories. Q1 had its own repositories hits on repositories. If we were working in different modules on different components we had our own. And one of the things that I like and I liked of artifactory was that it was super efficient how they store the data. So, as I said, we had tons of repositories. Some of our shared dependencies existed in several repositories. But the storage never grew out of hand. Because what JFrog does and artifactory does, is stores the dependency based on the SHA. And what the other repositories do, it's only linked to it. So, later on if you're using your repository, artifactory or artifact repository for promoting and following how your binaries have changed inside your company, well, it's super easy. You only move it to another repository and it didn't take time. So, if we had an application that had a lot of media, like images or movies, and they were vetted by whoever needed to vet it. And later on, they created a package and sent it to production. We didn't copy the entire resource file, like, several gigabytes. No, we only created a link in the repository. So, that's why virtual repositories are so important, because if you have that level of segmentation and control, later on, it is easy to create this virtual repository with handpicking or cherry picking, what you really want. So, and again, the top of the virtual one, you have the list, and there was some difference that I will explain later on of all the action that you can perform on your repositories.

So, let's continue. We're going to configure our npm repository. So, as soon as I create the npm repository, it will ask me, here is your npm repository, enter your name, and as soon as I do that, it creates and it actually shows me what are the configurations that I need to use in my project to use a particular repository. So, that is the URL that I have to use for getting my dependencies or to publish my dependencies. and I copy, check the first command, npm config, set the registry to the URL that I just got from my browser, and I do a npm install. And I have an error. Oops. I didn't authenticate myself. So, I go down in the same screen and there's the authentication and actually if you see there, it says, it doesn't show you your credentials. But if you go up and enter the password of your account, that particular snippet of code will have your credentials. Of course, I'm not showing you that right now because it shows my token, but believe me, you will be able to do that in your own instance. So now that I, again, did it to my npm configuration file and add that, I can run again my npm install, and this time I'm going to use the variable's mode. And as you can see now, well, in this particular one, we cannot see it. Let me see.

8. Resolving Dependencies and Publishing

Short description:

No. But I'm already resolving all my dependencies into my repository. The first time I do an npm package and target my artifactory repository, it takes time. We download all the dependencies required by this project into the remote repository. Then we publish the package by changing the name and version. It calculates the integrity and uploads the files. Finally, we scan the package for security by selecting the x-ray option in the MPM local repository.

No. But I'm already resolving all my dependencies into my repository. Oh, yes. Probably you're using the same authentication. Because you are actually, sorry. Before I confuse everybody, there was a question in the chat, and it says I did not get this authentication error, but I'm logged into my GitHub. So chances are that you have already configured your authentication tokens from GitHub. So we're borrowing them. Well, no, we are not borrowing them in terms of we're knowing what you're doing. No. You know that there are protocols that we have like a pass-by token. He says that you're okay, so we trust him and he trusts you, so we end up trusting you. That's the idea. Not I can see your credentials for GitHub. That's not the idea. So anyway.

The first time that I do an npm package and I'm targeting my artifactory repository, it takes a lot of time. So don't get too excited about that. It's because what is happening is we are targeting the remote repository and we're downloading for the first time all the dependencies that are required by this project into that particular repository, the remote repository. And this is exactly what you're going to see. Now you have that happily serving 400 and something artifacts. And if you double click now, you will see in the remote repository all the packages needed by this project.

The last time that we're going to do is publish. But as you may think now, this is super trivial. I have already my repository. I have already my credentials. So what is the actual complication here? Actually, there is none. So you do only NPM publish. In this case, because we are actually using another person's open-source code, what I actually did when I edited the package JSON is changing the name and changing the version. If you don't change the version and you try to publish, well, the same version already exists. So it should tell you no, no, no, you cannot publish against the same version. But so in this case, what I'm doing is changing the name and the version so it's very clear what we're doing. And when I run it, it says, yay, you did it. And as you can see, it's calculated the integrity and tells me how many files it actually uploaded. And as you can see, I change it to DevOps JS dash Rtrim instead of the original name. And now if we go into our local repository, we can see that there is our published artifact. And well, I did it I updated all this screens in the morning because I always like to run the same instances as you will see and you will experiment in this workshop. And we're going to go into the final part. And the final part is scanning for security. Remember all this idea of we should trust a little bit more our dependencies. So I go into our MPM local repository and I will select the x-ray option. So that means that this particular repository will be scanned with x-ray. And then I go into the x-ray tab and check if it's there. If it's not there, I have to add it. And when I click add repository, it will show me all the repositories that I have that are not already indexed by x-ray. And you can add it. There they are. And later on, I can execute different functions for each repository. And one of it is start indexing. And voila, we scan our package.

9. Dependency Management and Project Setup

Short description:

And x-ray tells us that it's scanned and there's no problem. Later on, I can install it again. We have several ways to keep track and enrich the knowledge about dependencies. It is a shared responsibility across the entire company. Now, we can do it together. Projects. The first thing we are going to do is clone the project. This is the URL of the project. I'm going to stop my share because I'm going to delete my NPM RC. Let me change my NPM. We have everything like that. This is NPM install. We are resolving everything from NPM registry. So, there is no magic here. Let me share my browser now. Because now we need my browser. But first of all, I need to open my browser. Which is...

And x-ray tells us that it's scanned and there's no problem. Later on, I can install it again. I go to a different folder, obviously. And MPM install DevOps JS. And it will happen what we expect. And I didn't add the new slide. Well, obviously after you install it, it will change on the upper part of your screen, it shows you how many downloads do you have from this artifact.

So we have, as you will see, there are several ways to keep track and to enrich the knowledge that we are creating our companies or in your companies about this state, the functionality or any idea that you have close to your dependencies, you can annotate them, create metadata. So for example, you can at some point create a test and say this particular dependency doesn't mess up with our performance. And later on, you can say show me all the dependencies that have been tested for or have this flag. And then it's not only the responsibility of your team of keeping track the dependencies help. It is a shared responsibility across the entire company.

Now, we can do it together. Projects. So, what I have here is the I'm going to remove everything. So, nothing under my sleeve there. So, the first thing that we are going to do is the clone. So, I'm going to clone again. And I'm going to put it in the chat. So, I, hi! So, I'm going to clone the project. And this is the URL of the project. So, if you want to use it. I mean, it is a project that you want to use. This is something you want to follow. It is just installing the dependencies, configuring your repository, we will create your repositories and publish a package. So, anything can work. So, now I go back again to R3. And there is nothing like this tree again. This is very similar to what we saw. So, that's it. I'm going to stop my share because I'm going to delete my NPM RC. And I hate that there is no way to show in a script or in a plain text. This is my secret. You shouldn't it in plain text. Let me change my NPM. So, I only did like common to all of my registries and my authentication and everything. All of the preferences that I have for artifactory. So, if I do an NPM install and lock level variables. So, I like to use lock level variables because it shows who is resolving your dependencies. So, again, the other thing that I should have mentioned that it actually works with a scope packages, but when you're creating your repository, you will see that that's the case. So, nothing, it took five seconds and we have the same thing as I showed you in the screenshots. So, that's perfect. We have everything like that. I mean, this is NPM install. We are resolving everything from NPM registry. So, there is no magic here. So, let me, again, share my browser now. Because now we need my browser. But first of all, I need to open my browser. Which is...

10. URL Provisioning and Repository Creation

Short description:

And you need the URL. Has everybody provisioned their own instance? Is there any problem? Nothing? I'm not sharing... Well, I can share now. Sometimes Zoom is surprising. So, okay. So what I can do is delete all my repositories. But let's see. Actually I can remove this one. And as soon as you start creating the repository prefix, you will get like the local one, the remote one and the virtual one. And as soon as I create it, this is what I already told you. This you can select the different ones. Here now I have all the information for this one, this particular one. And if you copy this code, you will have the URL. Again, if you enter your credentials here, you will have your own username and password. And what happens if you close this window? You go again and select this one and you go into set me up. The first thing that I suggested you to do was in identity and access add your own new user. If you add a new user, you only have to say email a password, the roles that you want. And here's what it's starting to get interesting. You can start creating groups.

And you need the URL. Has everybody provisioned their own instance? Is there any problem? Nothing? I'm not sharing... Well, I can share now. Where is my browser? Wait a second. Sorry for that. Keynote. Where is my browser? I don't have a browser. I'm seeing a browser. Wait a second. Yes. Now it's. Sometimes Zoom is surprising. And I'm not sharing my entire screen because I have an insanely huge monitor, insanely. Not my idea, my husband's. But I shouldn't complain. It's actually very, very handy. So, okay. So what I can do is delete all my repositories. But let's see. Actually I can remove this one. So as I said, we have the different artifacts. The different artifacts are going to be grouped here. And as you can see at the top of this page, I have the virtual ones. Then you have the local ones. And then you have the remote ones. And then you are seeing the remote cached. Because obviously this is not going to resolve or store any artifact until you actually need it. And you can have your own policies for when to delete each one of it. So again, remember if I want to do a new repository, I can click here. You select your repository type. You say that you want a new one. And here is TEMP. I'm going to create only the TEMP. And as soon as you start creating the repository prefix, you will get like the local one, the remote one and the virtual one. So you can follow your own conventions for creating it based on your teams or your stage or whatever you need. Let's go with this. And as soon as I create it, this is what I already told you. This you can select the different ones. Here now I have all the information for this one, this particular one. And if you copy this code, this is the code that we have to copy. You will have the URL. Again, if you enter your credentials here, you can you will have in this part the snippet code your own username and password. And what happens if you close this window? You are like, oh my goodness, where is that configuration file? I know, don't worry. You go again and select this one and you go into set me up and that happens again. So that's also really important to do. And I actually didn't even follow my own script. The first thing that I suggested you to do was in identity and access add your own new user. If you add a new user, you only have to say email a password, the roles that you want. And here's what it's starting to get interesting. You can start creating groups.

11. Configuring Repositories and Authentication

Short description:

You can start restricting access to repositories and configure authentication in the npm rc file. The quick set me up option allows you to create different types of repositories, including Docker, Maven, NPM, and more. It is important to have a single source of truth for resources, ensuring they are versioned and released correctly. The npm registry provides information about dependencies, including access rights and customizable properties. These tools make it easy to enhance and annotate dependency information.

You can start restricting who has access to what. So we already can have this level of granularity. I'm going to close this because that annoys me a little bit. So if you return to the first step, you have the artifactory and again, discard. You will have your repositories there. And again, if you want to see the configuration, parts of every repository, it's here. Is that clear so far?

So again, the next step is to copy your authentication, editing your npm rc file and run it. When you run it, let me uncomment again, my npm rc and change again the my share and I will share now again my. Okay, so now I have the N modules, I'm going to remove them again and again I'm going to install it. Well, in my case it's going to be super fast 2 seconds instead of the 7 that you probably have seen and that's the reason because I didn't change or delete the repositories. I could have done it but it's okay. So here it's telling you what is the URL that you're getting. Actually the DevOps test in PM. Is that everybody is following me? I already lost somebody. Please do tell me. Let me check my the chat. Sometimes I miss that because the chat closes every time I change the share screen.

Okay. So where am I creating the repositories? For example, if I, for some reason, I want to create another repository, I can go to the quick set me up. the quick set me up shows you all the package types and you can select what are, actually, the most typical ones that we have is Docker, generic, Maven or NPM. But if you can see, we have also, so, you name it. PHP, we have Go repositories, so there is a lot of package types and we, of course, have the generic one. And because, again, as I said, one of the dependencies that you can have is the resources. And resources can be media files. Whatever you want. Because sometimes they also have to be versioned, they have to be released, they have to be added to a specific version of an application. So, it is important that we have a single source of truth and the single source of truth is using the same configuration for security and rights. Let me look again at my chat. So, in case somebody is asking me questions. I'm not liking okay. Perfect. Don't worry. Yes. You can follow the recording whenever you want. So, let's go back to the here. It's only the repositories where you have seen. And this is something that we already saw in the DevOps test local. Well, actually, no. What I want is to show you the test npm. Here is where I have all the dependencies of this package that were created or refreshed based on that npm install that we have just done in our in our terminal file. So, if you go into here, you can have all the information that we have available that is coming from the npm registry. Some of these packages tell you I mean, this is some is almost the same information that you have in npm packages and it can show you who has access over what thing and what properties you get from this particular one. And this is the part where I said that this kind of tools make our lives easy. Because if for some reason we want to annotate or enhance the information that is available for this particular dependencies, we can create our own property name with our own property value. And the property value can be text, can be numbers, can be numeric, can be a specific range. So you name it. That is super powerful. Okay. Let's continue. And I will share again my terminal.

12. Publishing and Repository Artifacts

Short description:

When publishing to the repository, I change the name and version directly in the registry. The new version appears in the repository with the correct information and is being indexed with X ray. The package is scanned for violations and no issues are found. Multiple artifacts with the same name can be seen in the repository.

Okay. So when you execute this, you automatically are populating your remote repository because it has happened for the first time, the request. So now you have your cache, your dependencies. What I'm going to do now is we're going to change the package JSON. And here I'm going to call it, we're going to deploy a new version, which is going to be devops.js5, and we will call it devops.js the new version, because why not? Okay, devops.js. I opened another devops. New version. So it's very clear. I'm not going to change everything, nothing else. So the read me is and everything remains the same, so don't get confused. So I already need the npm package. I change, let's review. I only change the name. I only change the version. So when whenever I'm publishing to my repository, it is easy to do that. My link to the repository was found and when I change back to the to the browser, I will show you where do you get that URL and what it's actually you have in the deploy tab. Select your npm local repository. Go again to set me up. The window appears showing you all the different configuration files. If you go to the second tab, there is the deploy. You can use the registry where you want to publish your package. So and for my curiosity, go again to set me up on the virtual one. And again, if you go to the deploy one, you will see that it is the same URL. So now I'm going to do only the npm publish. But in this case, I'm specifying not in the package JSON, I'm publishing directly into the registry. Because I didn't modify that on the package JSON. And as you can see, I have a yes, version 1.65. With the short form and again, five files. And again, I'm going to share the order screen. This time it appeared at the first try. Yay! Okay. Sorry. It is because it took me a while. So, let me refresh. So, now, we have as soon as I refresh, now we have the DevOps.js, new version. And when you click here and you open and expand, you can see the MPM version is the 105. It has all the different information. And if I go into the repositories in the second tab and see the repositories and I go into the remote one. Remember when I was telling you that this was possible in the configuration file in the repositories, check which ones do you have if they were listed by different types. This is the place where we are doing that. And the most important one is this one in my remote. Let me expand a little bit. Hopefully that didn't mess up nothing here. So, if I select again, my DevOps this one, here's where you can enable the indexing with X ray. So, this particular repository is being indexed, and if you return here in your first, in the second one, you can see the settings and the watch, the policies. So, if you now go into your artifacts, you will see that our latest, this one, MPN, no, sorry, sorry, sorry, sorry, I'm not reading correctly, my local one, here, pass, it's being scanned, and there is no violations found. And here I can see the license. So it's okay. So that's perfect. So now if I check here again, because you can select here also, you can see that I have two different artifacts with the same name.

13. Creating and Configuring Repositories

Short description:

In this part, we have created different types of repositories, including local, remote, and virtual repositories. We have discussed the configuration options, the ability to run security scans, and the ease of publishing. It is recommended to use specialized package types for better performance. The speaker encourages the audience to ask questions and provides an opportunity for feedback. The part concludes by mentioning the speaker's willingness to assist with any inquiries or suggestions.

Well, in this case, it's only the JSON file, so it's okay. And you still can see the same information. So what we have done, we have created the different repositories, three types of repositories, three versions of repositories, local, remote, and virtual. We have seen that there are different types of repositories, almost all repositories following the same pattern. You can configure it, you select the different writes, you can specify the granularity to the level that you want, you can run security scans on them, and you can publish really easily. And once you have a virtual repository, you can create new virtual repositories, where you aggregate different repositories. But it has to be the same type. I mean, all MPM, all Maven, all PHP, but all the same, or all generic. I don't recommend you to use generic and mismatch different types of generics. First of all, because you will lose a little bit of performance if you're not using the specialized package type. Again, as I said, they are fine-tuned for the specific package type. So go always with the native. And I don't know. What else? What else can I show you? Well, I can actually show you a lot, but I don't want to start anything that we cannot finish right now. So is there any question so far? If not, I will show you the last slide. And I will let you go then. So as I said, I spend a lot of time giving sessions or working with customers, but I always ask feedback from my colleagues and, of course, from the attendees to my session. So please, please, please reach me out if you have any questions or comments or suggestions.

Watch more workshops on topic

DevOps.js Conf 2022DevOps.js Conf 2022
152 min
MERN Stack Application Deployment in Kubernetes
Deploying and managing JavaScript applications in Kubernetes can get tricky. Especially when a database also has to be part of the deployment. MongoDB Atlas has made developers' lives much easier, however, how do you take a SaaS product and integrate it with your existing Kubernetes cluster? This is where the MongoDB Atlas Operator comes into play. In this workshop, the attendees will learn about how to create a MERN (MongoDB, Express, React, Node.js) application locally, and how to deploy everything into a Kubernetes cluster with the Atlas Operator.
React Summit 2023React Summit 2023
88 min
Deploying React Native Apps in the Cloud
Deploying React Native apps manually on a local machine can be complex. The differences between Android and iOS require developers to use specific tools and processes for each platform, including hardware requirements for iOS. Manual deployments also make it difficult to manage signing credentials, environment configurations, track releases, and to collaborate as a team.
Appflow is the cloud mobile DevOps platform built by Ionic. Using a service like Appflow to build React Native apps not only provides access to powerful computing resources, it can simplify the deployment process by providing a centralized environment for managing and distributing your app to multiple platforms. This can save time and resources, enable collaboration, as well as improve the overall reliability and scalability of an app.
In this workshop, you’ll deploy a React Native application for delivery to Android and iOS test devices using Appflow. You’ll also learn the steps for publishing to Google Play and Apple App Stores. No previous experience with deploying native applications is required, and you’ll come away with a deeper understanding of the mobile deployment process and best practices for how to use a cloud mobile DevOps platform to ship quickly at scale.
DevOps.js Conf 2022DevOps.js Conf 2022
13 min
Azure Static Web Apps (SWA) with Azure DevOps
Azure Static Web Apps were launched earlier in 2021, and out of the box, they could integrate your existing repository and deploy your Static Web App from Azure DevOps. This workshop demonstrates how to publish an Azure Static Web App with Azure DevOps.
DevOps.js Conf 2022DevOps.js Conf 2022
163 min
How to develop, build, and deploy Node.js microservices with Pulumi and Azure DevOps
The workshop gives a practical perspective of key principles needed to develop, build, and maintain a set of microservices in the Node.js stack. It covers specifics of creating isolated TypeScript services using the monorepo approach with lerna and yarn workspaces. The workshop includes an overview and a live exercise to create cloud environment with Pulumi framework and Azure services. The sessions fits the best developers who want to learn and practice build and deploy techniques using Azure stack and Pulumi for Node.js.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

React Advanced Conference 2021React Advanced Conference 2021
19 min
Automating All the Code & Testing Things with GitHub Actions
Top Content
Code tasks like linting and testing are critical pieces of a developer’s workflow that help keep us sane like preventing syntax or style issues and hardening our core business logic. We’ll talk about how we can use GitHub Actions to automate these tasks and help keep our projects running smoothly.
DevOps.js Conf 2022DevOps.js Conf 2022
33 min
Fine-tuning DevOps for People over Perfection
Demand for DevOps has increased in recent years as more organizations adopt cloud native technologies. Complexity has also increased and a "zero to hero" mentality leaves many people chasing perfection and FOMO. This session focusses instead on why maybe we shouldn't adopt a technology practice and how sometimes teams can achieve the same results prioritizing people over ops automation & controls. Let's look at amounts of and fine-tuning everything as code, pull requests, DevSecOps, Monitoring and more to prioritize developer well-being over optimization perfection. It can be a valid decision to deploy less and sleep better. And finally we'll examine how manual practice and discipline can be the key to superb products and experiences.
DevOps.js Conf 2022DevOps.js Conf 2022
27 min
Why is CI so Damn Slow?
We've all asked ourselves this while waiting an eternity for our CI job to finish. Slow CI not only wrecks developer productivity breaking our focus, it costs money in cloud computing fees, and wastes enormous amounts of electricity. Let’s take a dive into why this is the case and how we can solve it with better, faster tools.
DevOps.js Conf 2022DevOps.js Conf 2022
31 min
The Zen of Yarn
In the past years Yarn took a spot as one of the most common tools used to develop JavaScript projects, in no small part thanks to an opinionated set of guiding principles. But what are they? How do they apply to Yarn in practice? And just as important: how do they benefit you and your projects?
In this talk we won't dive into benchmarks or feature sets: instead, you'll learn how we approach Yarn’s development, how we explore new paths, how we keep our codebase healthy, and generally why we think Yarn will remain firmly set in our ecosystem for the years to come.
DevOps.js Conf 2024DevOps.js Conf 2024
25 min
Atomic Deployment for JS Hipsters
Deploying an app is all but an easy process. You will encounter a lot of glitches and pain points to solve to have it working properly. The worst is: that now that you can deploy your app in production, how can't you also deploy all branches in the project to get access to live previews? And be able to do a fast-revert on-demand?Fortunately, the classic DevOps toolkit has all you need to achieve it without compromising your mental health. By expertly mixing Git, Unix tools, and API calls, and orchestrating all of them with JavaScript, you'll master the secret of safe atomic deployments.No more need to rely on commercial services: become the perfect tool master and netlifize your app right at home!