CI/CD 101 with CircleCI

Hello, everyone, and welcome to today's workshop at DevOpsJS. We'll be covering the topic of CI CD and do so with CircleCI. So, my name is Zan. I'm a Developer Advocate at CircleCI, and I'm joined today with my colleague Angel.

Say hi, Angel. Hello, everybody. Thanks for showing up and sitting in our workshop. So, appreciate everyone being here. I know there was a question in Discord about our Twitter handles. These are our Twitter handles. So, if you want to reach out to either one of us, we are here for you.

Thanks, Zan. Absolutely. Absolutely. Yeah. Any questions, we're happy to help. And, yeah. So, first things first. We'll be doing a few prize draws at the end of the session. It's going to be a challenge for you, essentially, as you follow along the workshop. You will give you an assignment, you complete it. From the people who completed, all of them will receive CircleCI Swag. And, we will also draw a few copies of the DevOps book by Emily Freeman, which is amazing. And, I'm not sure if this is visible, but that's the book. It's pretty cool. I learned most about DevOps that I know from this book. So, yeah. And, one pair of Sony XM4 headphones, which is a pair that's very similar to the one I have, but newer. And, they're arguably the best noise canceling headphones out there. So, pretty good. So, stick around, follow along the workshop. It's going to be a very quick to do assignment. And, yeah. We'll draw some prizes in the end. The cool thing is it's relevant to what we're going to be showing you today. And also, Jean spent some time getting this. Some thought and time behind this. So, I'm excited to see how this works out. It'll be fun. I'm hopeful it works out. It's going to work out. It's going to be fun. Come on. Yeah.

Anyway, we'll cover some theory first. So today, we'll be talking about continuous delivery, which is all about delivering changes of code to users using the tools that hopefully aren't manual and essentially are automated. So that's the whole wide-ranging practice of doing so. So basically, it's taking code, getting that code to the users with new changes built in.

CI, CD is like a subset of that. So, continuous integration is what CI stands for and continuous deployment is what CD stands for. Continuous integration essentially encompasses everything from taking the source code and building that, running tests, essentially whatever you need to do to produce some kind of built binary version of that code, but it's still not with the users yet. And then that's where continuous deployment takes hold and essentially makes sure that that build code gets to the users. And yeah, CircleCI is a platform that allows for that.

So yeah, that's exactly what I said. So it, the code doesn't really, like it's usually, we're working on it as developers will be using GitHub build tool, but there is like any other environment's build tools that people might, might use for, for source code, collaboration, VCSs. And then once you have changes happening that to that code, which is commits, merges, feature branches being in pull requests, open all that stuff. That code is then kind of triggered in a CI CD pipeline, which runs a bunch of things for us, which is, yeah, tests. We also do some vulnerability scanning and ultimately build some Docker images from it. But like this doesn't need to be just Docker images. It can be anything. It could, you could build static website. You could build, you could build a full on kind of monolithic application deployed wherever that might be running. Anything really. Yeah. And yeah, continuous deployment is, will we actually get that code from our Docker image in our case, to our users. And yeah, that could be essentially either like Kubernetes clusters to just like CDN that kind of delivers your statically available websites to mobile applications, anything can essentially fall under that continuous deployment category in JavaScript, in particular.

There is like a bunch of other things to consider because JavaScript is very, very, very, very complicated. Just because of all the tools, all the environments we have available for us. So if you just think that all the permutations and combinations of browsers and extensions that people might come up with on just the front end, from mobile to desktop, to all the OS's and all that stuff, all that needs to be kind of tested and a CI tool can help us do that. All the way to unit testing, which is purely in the JavaScript category, all the way to unit testing, JavaScript sites, JavaScript code for not only browsers, but also to cater for Node.js projects. I mentioned static sites, APIs obviously can be driven from Node.js and also serverless functions. That's very often done with JavaScript or TypeScript. And then you have all the build tools. TypeScript, I mentioned one, and all the bundlers like Webpack that are just kind of part of that very complex tool chain of JavaScript that is very hard to kind of understand or not understand. But it is complex and if you don't know exactly how everything comes together as in a single project, it's very hard to really fix one individual portion of it. And yeah, a CI, CD tool helps us codify that, we'll see that how we essentially codify that and package this alongside our in our case GitHub repository or any other kind of VCS provider repository that you might be using. So that anyone who comes to that project can actually just read through that CI, CD configuration and understands exactly what's happening where. I mentioned configuration so actually let me let me show you some some code first. So CircleCI is obviously a CI, CD provider that we'll be using. We configure everything from a single file which is config.yaml. Zoom in a little bit. So we configure everything from this config.yaml and basically tell it what to do when a commit happens on a particular branch. This one is very simple. We'll be building off that and just kind of adding more logic to it, adding more steps to it. But essentially that's one file that's committed alongside your project inside of your repository that kind of configures everything. And the other part is this dashboard that we have circleCI dashboard which basically gives us a view of everything that's been done with your project, with your project, how it's been built, what's been built, what tests have failed, what tests passed, all that stuff. And yeah, so at a top level what gets triggered each time you commit is a pipeline, which essentially contains a bunch of workflows. We'll be working with a single workflow but you have a multiple of that. And within that workflow you arrange, what you arrange are called jobs. Which are essentially steps to execute in a given manner. Like commands or steps to execute in a given manner in a specific environment. For example, a docker container. A particular docker container. So each job is like this kind of self-contained thing. We'll have jobs for building our docker image, we'll have jobs for running our test, running all vulnerability scans and a few others for example. And yeah, our workflow will essentially orchestrate all of these jobs, make sure that they're happening in the right order and under the right conditions. And yeah, each job can have a multitude of runs. So you can specify like parallelism for like running stuff in, we've had like a lot of functional tests for example, you can run just a subset of that in a single job so that you essentially run the whole thing faster, because you're going to split work across different parallel running jobs or you can have, or you can have stuff running in a matrix of various versions. For example, if you want to run browser tests across Firefox, Safari, Chrome, you could do that with like this kind of matrix that kind of runs all of your jobs at the same time. Angel, if you've got anything to kind of jump in with, please do. Otherwise, I'm just kind of rambling on. Oh yeah, just on the parallelism bit, yeah, it's just basically concurrency, it enables you to do multiple things at the same time, which is ultimately what you want, right? You want to be able to make, or yeah, you want your build pipelines to execute as quickly as possible so that you're not sitting around and waiting for the feedback loop or the results of that, right? So parallelism is again just a way to control the amount of concurrency within your pipelines, and hopefully that will get you a fast, lean, fast pipeline. That's all I got about that.

Yeah, anyway, let's talk about the more interesting thing. I see there's some questions and that was answered, I suppose. That's good. Yeah, if there's any more questions or comments please ask them in the Discord site so that we can have visibility, and then we can answer if we don't get time to answer them. We can do that at a later date because we want to get everybody recorded and answered. Once this Zoom goes away we won't have access to the chat.

Yeah, so please use the Discord. There is a follow-along repository. Let me copy this over. This here is a shortened link but this is like a copyable link that I just pasted in Discord. There was a question, did that differ from what GitHub Actions offers? So CircleCI is a CI-CD platform we just do that and we do that really, really well. GitHub Actions is more of a DSL on top of all the features that GitHub has, but one of those features, a very small part is a CI-CD. You can use it as a CI-CD, but you can use it for many other events that are useful in GitHub. So it's like more GitHub Extension toolkit, rather than a specific, dedicated CI-CD service or platform that CircleCI is. Is there any UI button integrate with GitHub Actions for trigger event on the pipeline? Not sure I get this, essentially- I think he's asking about, if you can trigger a pipeline bill, a data event on GitHub Actions from a pipeline. I think that's what he's asking. And the answer is there is no button because we do configuration as code, so you could code a trigger like a web hook, or direct API call, or even use the GitHub CLI in your CircleCI builds. So that's how you would do something like that. Cool.

Okay. So I will move on to my GitHub now. We have the source repository and the CircleCI public organization where we have a bunch of other public facing things. But what I'll do for the purpose of this workshop is I'll make a fork into my own personal GitHub account. Just to show you what's happening. Should they be following along at this point? Yes. Yes. Essentially, this is the part where you can start following along. If you're stuck with anything, please do let us know. We have a lot of time so we can definitely address any issues as we go along. And for people who have just arrived, we are doing a prize draw at the end. Don't forget about that. We'll be giving you an assignment at the end of the workshop where you get to build something yourself, show us, and then you can win a pair of Sony XM4 headphones which are amazing. You'll be entered into a raffle. It's definitely a prize draw. We have one pair of headphones to give away. We don't have a bunch of them. We have also a handful of copies of the DevOps book by Emily Freeman, which is pretty amazing. Everyone who completes the challenge will have CircleCI swag. So you always get something. Will you deliver to Germany? Yes, we do. John is located in the EU, so I'm in the US. So yeah, I have forked their repository, CircleCI public CICD Workshop JS, and this is a fork on my project. We have a little starting config that I kind of showed you earlier that's very empty, but it has a little bit of things. In this kind of dot circle, CI directory, which is in the top level, you'll also see a few other conflicts, essentially the steps throughout this workshop. So if you get stuck, if you have to step away for, let's say, 10 minutes, you can then jump into the later date after the workshop essentially. Or during the workshop, if you kind of missed a few steps, you can always get into a more later version of that. Anyway, so we have this config. I will now sign out of CircleCI, just to show you how this call works. If you go to circleci.com, which I'll be dropping a link to, if it's your first visit, your landing page might look slightly different. But essentially, towards the top-right-hand side, there is a login or sign up button that you get to click, which takes you to this login to CircleCI page. And what I do is essentially just log in with GitHub. If it's your first time using it, obviously you'll need to authorize CircleCI to have access to your GitHub profile. Now you'll probably enter your own project dashboard. And this is where in the future all of your projects pipelines are going to be visible. And what we want you to do is go to this projects tab and find a list, on this list your CICD-Workshop-JS, which you have forked hopefully. Jean, can I interrupt? Yeah. Go for it. So one thing that happens sometimes with new folks is, can you just click on your profile again? Because if you're a part of multiple organizations in GitHub, then you may not see the things that you're looking for, right? So can you click on your profile again? It's just something I like to call out, because it confuses people in the beginning. What we do is we pull in all of the version control platform, in this case it's GitHub. We pull in all of the permission structures from there. So as you can see, Jean, his one GitHub account Zmarkin is a part of all of these organizations within GitHub, right? So just so you know, I just wanted to call that out. So if folks are part of multiple organizations, like Jean here, you need to you know, figure out where the repo is that you've forked to and what organization that lives because sometimes people forget like, Oh I think it's imported it or forked it into, you know, an organization and they forgot and it's not visible. That's helpful. Just calling that out. So if you don't see it, you forked it and you don't see it, try to find it using this profile manager system. That's all. Thank you. Thank you Angel. Yeah this is a very important thing to mention. I'll also copy this again and drop those instructions in, go into Discord.

Yeah and in this Projects tab that would have been visible if I had more room on the side, you would see your repo. And what I'm going to click is Setup Project. So what Bjorn is doing here is associating the repo to your CircleCI account, if that makes sense. So what you're doing is basically turning on CircleCI's monitoring of your code. Exactly. And because in our repo we already have an existing config.yaml in this.circleci directory, it'll just kind of ask you if you want to reuse that config. If you did not, it would very much... very likely give you this exact same config as a suggestion, which is kind of our Welcome to CircleCI configuration, which we'll see in a moment. Once you have that, you can click Start Building. This now redirects you to this project's own view, and you see that our first pipeline, cicd-workshop.js, is being run. It has a workflow called Welcome, it has a job Welcome slash run, and it's tied to my commit on this main branch which is basically built of what I've been kind of messing around with earlier. Imagine that there is no git history. Yeah. What this does, we can click on this job, which basically does a bunch of things. It tells us which steps it's taking. First, it's spinning up the environment. It's starting a container, this welcome config image, and then it's preparing some environment variables. Keep in mind this one, it will come in handy later, circles circle underscore build underscore URL. This is part of the prize. This is part of the assignment, but we'll get back to you. You don't have to write it down right now. We're just teasing you right now. Then it basically gives you this, this job basically gives you congratulations, you can successfully connect to CircleCI. And it tells you a bunch of next steps where you can go for tutorials for documentation, that kind of stuff. So it's like a nice, nice ideas for how to get started. So hopefully your projects are all configured well, and we can go... Actually we can go like back to the top level and see that our status is successful. So there is a nice green checkmark next to it, which means that our build has passed. And we can now create some real config for this. Before we do that, sorry, should we make sure that everyone kind of succeeded? Before we move on. Is anyone stuck with anything? Because this is like your most fiddly bit I would imagine outside of like setting up different accounts and stuff like that. Yeah. Let's make sure everybody has it forked at a minimum. Right? So if you don't have it forked or you're having an issue, just raise your hand in Discord and we will get you caught up. This is really difficult because we're virtual. So we're trying to get everybody to the same place. If you fall behind, just hit me up on Discord because John will be leading most of this and I can try to help on the side, if it makes sense. Absolutely. Thank you, Angel. Right. What we'll do now is we will create a real config that does real things for us. That's not just like a welcome hello world thing. Can you please post again the Discord URL? It's right in the chat right above. Oh, maybe you can't see it. I can do that. Let me just, I'll repost it. Oops. I got it, John. You got it? Cool. Yeah. Thank you. Anyway, yeah, let's, we will now, we will now clone this project back to GitHub. We'll clone it with this code button. And, yeah, we will go now to Visual Studio code and we will open a different config, new window, which contains Visual Studio code as my editor of choice. And, I never know how to start the terminal. Command tilt. Not on a Mac. It's control, shift, and dot. Oh, no, it's control tilt, control tilt. That's so an issue. I don't know, at least on my Mac. Yeah, well, I've got a PC keyboard. So it's all a bit confusing. Anyway, so I cloned my own fork. Basically, why you need to create your own forks is because you will be pushing a lot of stuff. We'll be making a lot of changes, pushing more config, which will trigger further builds essentially. So that's why. Okay. Right, and I forgot to do that. Now I've opened my file.

So I have this config.yml. And what I'll do is I will essentially go to this one-config-basic.yml and copy everything over to config.yaml. We're still in the.circleci directory. Which is where all the circleci config is. And I'll make a commit and then I'll poke you through what this does.

First steps. I'm using the main branch of my own projects kind of build everything. We could be using a fork, it would still build everything regardless. Okay, so I have pushed some more code, we can look at circleci, see what's happening. And it's already been triggered, it's already been triggered, it's already kind of building things and failing, good. And something is not right, guess what's wrong.

Now, if something fails in CICD, obviously you need to understand why and what's gone wrong. And the easiest way to do it is to just kind of drill into the job that's telling you that it failed. For example, that's our built and test job. Obviously because everything is tied to commit, we can just kinda open that in the tab and see, okay, these were the changes that we made. But you can also just kind of go through the job and see where the failure or where the issue happened. And my issue is essentially in this copy test results for storing step. Cannot create regular file, not a directory, guess why is that?

Angel, how would you suggest we fix that? Yeah, so you're obviously missing something, we have SSH capabilities, so since you're running the code in an executor on our infrastructure, its CircleCI infrastructure, I would re-run this job with SSH. So that you can debug in the environment where it's failing. Unlike any other, most other CICD platforms, they don't have this capability where the developer can jump into, or sign into, or log into the executor, and then do some live debugging. So you don't have to bother in operations person or a security person to gain access to these pipeline executors or run times. So, yeah, let's give that a shot. And do some live debugging.

Oh, someone said, darn, I did not. Lol, okay. Huh. That's not good, because this failure was actually scripted. Did you copy the whole thing over? Someone didn't have it failing. Angel, can you, maybe, help that person out, and I will, kind of, start talking about this? Christopher, jump on Discord real quick, we can debug it. You probably copied the wrong file over, I would say. Try, which file, show them again which file. Yeah, so I copied over this one-config-basic. We started off with the zero-config-start, and we copied over one-config-basic. Yeah, so, basically. So, start is the one we already had, yeah. All right, thanks for that. So, what we do is, because something is going on in our environment, we can actually use this re-run feature. I'm gonna build and test job that's failed, and I have this re-run with SSH button here. And we'll see what happens. Do, do, do, do, do. While that's kicking off, I'll explain a little bit about that feature. So, with the run job re-run with SSH, it's gonna run the job again, and then it's gonna fail again, right? It should fail again. Once that happens, the runtime that you kicked off will stay alive so that you can SSH in. So you'll see in a minute, in a few seconds here that the job will stop, and it will remain alive until two hours. If you access it, it'll stay alive for two hours. If you don't access it, it will terminate itself in like 10 minutes. You know what I mean? So, there it is. Yeah, one person asked where this failing thing was. So, if we were in this view first on the project view, you could see on the job side when it failed. I clicked on this build and test job that failed, and then the rerun button is on the top right hand corner, and you kind of get a dropdown. Basically, that's if you rerun an individual job, you don't rerun a workflow, basically that's what it is. Okay, so... No, you have to go back. And I broke it? No, what happens is if you click out of it, it will refresh the, so you have to go to the jobs type in that URL, you have to click jobs. There you go, you got it, it's right there. Yeah, so jobs, they have like sequential numbering. So I'm clicking now on the enable SSH job that's running. And basically it gives you a line to copy. Now on this. Or you could go to the bottom too like show them that as well. Oh yeah, absolutely yeah, you can also go in the bottom. I would actually use the bottom one, yeah, because that's... Even though they're the same, but. So what this happens is I'm now SSHing to this machine and basically inside the running Docker container. So if I use this SSH feature for my project, then what I need to set up to enable SSH in, essentially you just need to have a failing job and then you can kind of run each job with SSH and then debug it from there. It's primarily a debugging tool. Yeah, you can run this whenever it doesn't have to fail either. So if you just want to look at results or just run some things, you could do that as well. No, some folks are, yeah, have happening, no SSH key issues. Yes, so this would be a GitHub settings thing.

If you go to your profile, you have SSH and GPG keys. You basically set it up on a computer that you're using so that, because you're essentially coming from your local developer machine and kind of associating it with, we're not gonna go into how to create this, but essentially they have a pretty nice tutorial how to create, how to generate everything it and that's how you edit. I'm dropping the link to the GitHub SSH tutorials here. They will tell you, it's all guided essentially a tutorial. So it's like quite straightforward, but yeah, we don't, I don't think we have the time to have everyone set up on that, but it's good feedback. We have not thought of that, that it might not work, but I'll still show you what's the debug process like. Yeah, this is not critical to the workshop because you won't be doing this. Don't worry about the SSH. This is just us showing you what you can do, okay. But yeah, once you're finished with the workshop, you can follow along with Shawn's links in Discord. And yeah, let's ask our questions in Discord please. Cause I'm jumping between two screens, it's really hard to, and then we have also, and then we have also track, right? We can track people's questions. So thank you. Yeah, so essentially I have, I have my config open on the screen. So I see, I will have my config. I have my config open now and I see, okay, this copy test results for storing is the name of the step that failed. And we can go into this project now and this is where it's actually kind of checked out all the code. This is exactly the same, but obviously within the Docker container and I can see that test results XML is existing. I can do, I can do a cat to just see that the file is, file is there so it's running some, it did run some tests. So my problem was in this, do not create regular file, not a directory. Was that the exact same issue we had? Yeah, that's the same error. So I've replicated it from within the shell and it's because my test results doesn't exist and we need to do mkdir to create it. And now I might be able to copy my test results over. And it has passed. So basically what I've done is I've kind of given you an idea of what a kind of possible debug process might be. And because I have figured out what needs to happen, I can essentially like move that command that I forgot to add earlier, this mkdir test result, gonna create our test results environment into my config so that circle CI will execute that before trying to copy it so that our test results will never fail. With exit, I can escape that, I exit that SSH session and I can do, and I'm pushing that back to my branch. Cool. Now back to the regular programming. When I'm at this job, I can actually just hit cancel job, which will now close it. Now I can see that my process is now running and this is gonna run all our tests and see what happens. Let me go back to the config that we have and explain what it's doing. So we have two jobs. One is called build and test. The other one is called deploy Docker. As I mentioned, sorry. John, real quick, there's a question. Yeah. What did you just add and commit? Yes. I have added a makedir test results line to this. I will paste it into Discord. So, I've added the makedir test results line, which is now, obviously building successfully because that was the directory we were missing when trying to copy it over. So, building test is working. Deployed docker is failing. We'll get to that in a moment. Let's, yeah. So, and yeah, these steps, essentially run whatever you could run from the command line. Checkout is like a CircleCI specific step that basically checks out the code at this commit you have, but then the rest is just like run commands. So, run npm install. That's what you would run in any node project, on local, on anywhere you might run it. So, you wanna install the dependencies, you do npm install, and then we run a testci job, which is again, it's on our, this is the package.json, just like running my mock at test with this junit reporter essentially that I showed you earlier. And then we're doing all these kind of test, test processing to kind of store them. We'll get to these in a moment. And then this is the second job that we're doing. We're deploying Docker. Again, you tell it what environment to use. So node 15.4 image running in, it's a Docker image from circle CI. So it has everything you might need and again, just a command doing a bunch of Docker rethinks like build an image, tag it, and then push it to Docker Hub. At the end, we have a workflow which basically runs both of these jobs. First build and test and then deploys Docker because that requires build and test. So we're kind of orchestrating one after the other. So we want the test pass first before we are deploying stuff. Sorry. Can I jump in real quick? Yeah, absolutely. Back to workflows. Yeah. I just wanna call out that. And I wanna make sure everybody understands as well before we move on what these things are. I know we've touched on them in the slide deck but when you have a workflow, a workflow essentially what it does is it controls and or orchestrates how those independent jobs are run within a pipeline, right? So you could have multiple workflows that could use a mix and match of the jobs that you have up top. So these are just lists of things.

A job and a jobs and workflows are just a list of, right? Objects, which are job objects and then a workflow object is just something that orchestrates jobs. So that's why you see, Jeanne has one workflow called Node Test and Deploy. And then as you can see, there's a jobs key under that. And that jobs key is again, just a list of jobs. And as you can see what he's doing here is basically saying by default, CircleCI will run the workflows concurrently, right? So that means every workflow will execute on its own with no ties to anything. Now, if you want to build a dependency, meaning if you want things to run sequentially, like that line that Jeanne has in his, if you look to the right, in the CircleCI dashboard, you can see that line connecting the two. That's what the require statement does, right? It says, hey, deploy Docker has to, can only be executed once build and test, which is that require statement has been successfully executed. Is everybody clear on that? But I just wanna make sure before moving on because sometimes there's some confusion. I can show you the difference. Yeah. So if I commit that now, they'll execute at the same time. So basically I've removed the requires, requires line. But if the two are not dependent on each other, you can have them as separate workflows, maybe notice and then the separate one for deploy. Yeah, that's what I'm doing right now. So they could run separately those jobs. No, but I think he's asking me, you could have two workflows. Yes, you can create another workflow and you can say like node, let's just say no deploy, right? And you just do the deploy Docker job, that's it. But both of those workflows will run. Regardless, right? So it's just, your pipeline has two workflows. Yeah, and this is the one that's kind of running in concurrently with my new latest changes, you see that there is no line connecting them. And I will say that I recommend wherever you don't need dependencies, run it this way because it will make your builds faster. You know, yeah, eliminate as much as possible. And it's gonna be in situations where you need to build a dependency, but if you don't need to, get rid of it. I just like really fast lean pipelines. That's my recommendation for everyone.

Okay, so my deploy Docker job is now still failing. Let's see why it's that. I will make this across the whole screen so it's easier for everyone to read. Okay, my invalid argument, something, something, something. Invalid reference format. I know exactly why this is, but the error is not ideal. I know exactly why this is, but the error is not ideal. Essentially, if you look at this build, this build command is docker build T, because it's docker build issue. We're passing in docker login and image name and docker build image name, essentially ends up being just CICD workshop J S, which is our circle project repo name because our docker login is not set. What we should have really had is something that checks that our docker login exists. It's an environment variable. We have forgotten to set. And the same for our tag. We're using that. And docker password is another one that we would have encountered in the next step. Considering you can have the jobs run async that way when you would prefer that or having multiple workflows because you can also set. Yeah, you can also have workflows run async. I have yet to encounter something where I had to create multiple workflows. If not all of my examples actually use jobs in a single workflow that are going to run concurrently or separately.

Angel, why might someone create multiple workflows? Multiple workflows. So multiple workflows... Essentially, if you have, let's say you have a staging environment or a QA, different segments of your software development processes or cycles. If you have a development segment, all your developers care about is making sure you have a couple of unit tests run, making sure that the code they've just written is pretty tight, as tight as possible. The next phase typically is Q&A or a segment of your process where you would run more extensive testing on things. And then maybe you have a QA segment. So you could do multiple workflows at the same time and you can use a filter, which we'll get into later, that will help you understand how to promote things to the next workflow or how to execute the next workflow. So I think your name is Peter. I see your question and we, I think we're gonna touch on this a little bit later. So if you can be, yeah, we'll hit that a little bit later. And Jean, I think you're just missing your environment variable, right? Yeah, exactly. I'm missing my environment variable. So let's fix that. So that's why I said you need a Docker Hub account. Essentially it's another free account to create. It takes, what, a few minutes to set it up if you don't have it. But if you have a Docker Hub account, you need a login and password set as environment variables. And this step is gonna be successful. Who said? Sorry. Does everybody have a Docker Hub account? You're gonna need this in the future. Maybe give him a few seconds to. Or does anyone does not have one? If you don't have one, please just take a few minutes to set one up. And Jean's gonna walk you through. I'm sorry Jean. Yeah, yeah, yeah, exactly. I'm gonna log out and I'm gonna do a create. Essentially do. Boo. And. What? But that work so. Mike wolf I see a question about you, I'm assuming your time, but your Docker help account all you need is just to capture your password and your username for the next step is John's going to get into that in the SEC you can select the free plan you want or you can you can just scan your Docker account. Great to say. Um, you can select the free plan and obviously now I need to Verify my password, which I'll do. Uh, verify my account, which I'll do in the email. Um, yeah, you got an email verification and then you're in here and. Okay. That's for right now. All. Yeah. All they need is a password and username for now. Yeah. That's my username and my password is stored in my password manager, which you should absolutely be using, um, is someone's still, uh, setting up their, uh, Docker accounts. If you are, it's fine. We can wait. Cause this is, this is the important part where, yeah, we'll be relying on that for the future steps. So for now, we're going to keep them, uh, set up in, uh, with the same gear that I set up earlier. Um, so we're going to keep them, uh, relying on that for the future steps. Warren, uh, we'll, we're gonna get to that in a sec. We just get a little, uh, where you add the credentials. That's the next, uh, face. Why don't you just go ahead John, and, uh, if anyone, if anyone has any I will try to walk you through it.

Yeah. Cool. So, okay. So we need a Docker underscore login and Docker underscore password. Um, the credentials we add them in a scroll top to the, of the project, you have those kind of, uh, three dots you click project settings. This'll open another model menu where you have a bunch of settings set up. So we're looking, we're interested in the environment variables and you basically add your environment variables. So Docker login, and it's going to be. Uh, it's going to be this one, whatever your username is. Um. Passing that. And Docker password. And that's going to be seen on the other screen for me. Boom. Here we go. So I've added Docker login and Docker password. I can close that now. And what I can do, because I haven't changed anything in the project, in the repository, all I can do is just hit rerun workflow from start, or even from failed, which will just run the failed job. But I'll do from start, so it runs the whole thing. Yeah, two other accounts, you might need are Snyk and Heroku. Yeah, I'm gonna paste the links to get that going. Yeah, if you're done with the Docker stuff, you might want to get into those two. But essentially, this is now building my Docker image, doing all the tagging stuff with my new environment variables. After this step, you see that everything has kind of the building test has run successfully. It has completed. We will go to the building. After this step, After we have finished it, we will go back to the project settings into environment variables, the books are still kind of setting everything up. No, you ran that one, which is without requirements. It doesn't really matter. So yes, I ran. I ran the workflow that didn't have the required step. It doesn't really matter. The future step, we're going to have a brand new workflow that we're not going to build on the same thing. We'll be just copying over the next step. So it's going to be easier for everyone to just get back on the same level. But yeah, absolutely, you can do it either when they're running in parallel or sequentially, those two jobs. It doesn't really matter. Now that the environment variables are set, they're set for a project. And that means that whichever job is running in that project, all the following jobs are always going to use the same environment variables. And those environment variables that you store into your CI CD are never going to go into your GitHub repository, which is a very good thing. Yeah. Shall we make a very quick five-minute break before we go on to the next step? Set up remote Docker Engine. Can you, in Discord, can you send me the link to your circle CI, you know, the job that you're running? I can take a look and see what's going on. Yeah, because if the jobs are essentially running, if the jobs you have are public, you can essentially, like, share the… are based off public repos, everyone can also see the circle CI jobs because they kind of use the same access rights. But if your repo is private, obviously you can't share it to anyone who doesn't see that. That's the beauty of logging in with GitHub. But it's running now, it's just… Sometimes it takes a second for it to spin up. Yeah, okay, so the remote engine was taking a minute. Yeah. That shouldn't be, I'll note this and send it to our team. Occasionally, what happens is it's setting it up from scratch, and if your account has never run on CircleCI, this happens occasionally for the first time, then subsequent runs, it'll be a lot faster. So you've got to remember, we're allocating resources for you and a compute node somewhere, and sometimes things get queued up for initial runs. But once they're there, they're cached and then it'll be much faster in the future. But I see that it's running now, Masher, so should be good to go. Cool, cool, cool. Awesome, awesome. Shall we take a five-minute break? Or who wants to continue? No, I'm just kidding. Everyone wants to continue? We can do whatever. I don't know. I think people want to take a break, right? Let's take a five-minute and then. Yeah, if you're still struggling with environment variables, that kind of stuff, absolutely ask a question in Discord. We will be able to help you out. But, yeah, we'll continue at 7.04 GMT or four minutes plus the hour anyway. OK? Yeah. I will, yeah. Sorry. I was going to say, I'll post the links to the other thing. So if you don't want to take a break, you can sign up for the accounts that we're going to use in the future. And I'll post those links in a second in Discord. All right.

See you at four minutes plus the hour. Everyone still got their stuff working? Let us know please. Don't forget, there'll be a prize draw at the end with a little assignment and prize draw if you complete it. And everyone gets CircleCI swag, including this awesome t-shirt and a bunch of cool stickers.

Warren, I see that for the SSH, if you look in Discord, does John posted a link. So it's basically just have to generate SSH keys on on your local machine, like you said, so that when you SSH in it knows, right, that it's you are who you say you are validated by GitHub. Awesome. Awesome. Some is managed to set up environment travels, correct? Yeah, that's right. And it's green. Now. That's what we want to see. One of these. Yeah. So there is a question I need to answer here from from Peter is like to see both green and red builds just because like even if it's red, it's still telling you that it's still giving you a signal that something might be broken and you have somewhere to look at and fix strange I use my docker credentials, but still fail. Did he use? Just post the link, Michael. I sent you a thing. Okay. I can hook you up. Remo M. You can't find the, let me see. Uh, the Remo, did you click that link up there? Let me find, let me put posted again. Try that one. Remo. I like red builds. Tells me what I'm doing wrong. It's a signal that kind of tells you, okay, look into that. Green builds of boring. Yeah. They're boring, but they're good to miss. Yeah. I agree. That's just part of it. And I always miss it when I get results, but. If you're not failing, you're not doing it right. Or you might be, or you might be an Android, right? Or you might be NEO from the matrix. I mean, who knows there is no spoon.

Okay. I'm just looking at, so, yeah, we've covered, we've covered what workflows are, how to do how to orchestrate different jobs. We've kinda run a bunch of commands, fixed some of them through a sage. So that's a nice, a nice, very good solid foundation that essentially you can use to build any, any project on circle CI, or any project on CircleCI. And now in this second section, we'll be looking at how to make this way more interesting. How to make a build way more interesting and write a lot less code such as this. I'm not saying that this code is bad. I'm just saying that I don't like to read a bunch of command line instructions of bash in YAML because it just doesn't work on it. It just seems intimidating to me. So yeah. What we'll do is we will open this file config2-config-orbs.yaml, and we will paste that over into our config.yaml. So CI, CD is the best way to say you are bad at coding even if you know it already, absolutely. Or it's the best way to say that you're great at coding because you know how to fix your problems, right? Sorry, that's how I look at it. So if you copy that file over into that one, you will be on the way to success, which I'll do now. And we've got to write some bash scripts next time, man. I mean, there is like, we have all these configs prepared because YAML is notorious if you're trying to life code it just because of all the indentation problems, bash is like the one language that is harder to write correctly than YAML is, but it's guaranteed to work. Yeah, yeah, that's right. I was going to say yaml is harder to write correctly than yaml I think. It's just like, I don't know. I can never do it right. Okay. That's just, people are not used to it. That's that's the main thing, and even I write a lot of bash and I even forget. So like, cause there's a lot of ways to do things. You definitely have to learn some of that so you can. And this is just, I think, you know, the way it's going to be for a long time, developers who are getting into CI, CD or DevOps kind of functions, I would say spend some time and learn a little bit about the shells you're working with. Orb, sneak, not loading. Oh yeah, that's good. That's a good problem to have. What I'll do is, John, you continue, I will post a link to how to fix this. There is a... If you're having that problem. I can show you that on the screen.

Go ahead, show them. So on the, sorry, on the left-hand side, you have this... What's that called? The cog. It's a cog. It's called a cog. Under Security tab on the left-hand, you have Allow Uncertified Orbs. And... Make sure it's ticked to Yes. It's, by default, it's set to No, basically. So... Because orbs don't all come from us. Some orbs are made by the community. And some orbs are made by our partner organizations, such as SNEAK, for example. And, uh... Not every organization wants to... kind of, opt-in to having others' kind of code, essentially, that they can use. So, that's an opt-in step that you need to take. So, go into Organization Settings, Security, and click Yes to this, Uncertified Orbs. It sounds more scary than it is.

Now, we'll be looking at Orbs. First job has failed, which is good. And, we will be looking at what it's actually doing. So, looking at my workflow, I have two jobs again, build and test, and we also have one called dependency vulnerability scan.

I think all third-party orbs are uncertified. No, sorry, I think all third-party orbs need to be opted in. It's not that they're uncertified, because they are. Sneak is a third party orb, it's not from CircleCI. I can answer that. Certified orbs are basically things that CircleCI has built. It has nothing to do with security, so to speak, directly. Certified orbs just means we built them, so we maintain them. The partner orb, like Sneak, yes, we are partners, but it's just a different classification. They are trusted. We do some provisioning and certification, making sure that, or assessment. We don't certify our partners' orbs. We just make an assessment that it's valid, it's working, right? And then obviously the community orbs are the most dangerous because we do not, those are just publicly available. It's like open source MIT. Use at your own risk. So we give the ability for developers to the community to build their own orbs. And that's what a community orb is. So if you rate them, they certified and the partner orbs are good to go. The community ones are the ones that are like, they could be doing anything. We do have controls, but we can't catch everything. So that's how you would consider them. So don't fear our partner orbs. They are monitored by us and they work with us to ensure that they're working. The problem is they are still third party. And by the way, the certified designator, we are working currently right now to figure out a better way to get that across. It's not that it's insecure, it's just the way we labeled it when we started the program and we're working on updating that. I hope that answers your questions. Yeah.

My build fails and it's different from the one with Dan. Ha. Can you share with Angel your project? Yeah, just post a link into the thing and we can take a look. Did you copy this config that starts with a 2, 2 config orbs because that's the one that we want to be looking at. But essentially, we will be talking about orbs in this section. Which is why in this config, we have 3 orbs defined at the top. We have an orb for node, we have an orb for sneak, we have an orb for docker. and… Okay. Peter, you have to enable the orb. So for that, you need to go into organization setting, click on that cog, on the left-hand side, and go to security, and click yes to allow these orbs. You might have to click confirm, but no, apparently not. Yeah. Anyway, let's go on. So yeah, orbs are a great way to share common sets of config across different projects, across different builds. Organization admin. Are you using your, sorry, okay. Somebody need your mic please. You had to den... It's fine. Okay, so if you're having an issue with the admin, you need to run it under your own personal or either an organization or your own, What do you call it? User account on GitHub. Yeah. So that's, so that's what I suggest you do. So if you're running it in your company's organization or, or an organization on GitHub that you are not an admin on, you're going to have issues like that. So. Sorry, where were we? Yeah, here. Anyway. Yeah. So with Orbs, we can greatly reduce the amount of code we need to write for configuring our CircleCI. And inside of those Orbs, you basically can, can find jobs. You can find individual commands. You can find executors. So like a bunch of things that you can package and in this reusable format. So our now new Orb using config, we're going to actually move this one to the side. No, let me use this one. Move this one to the side, so you'll see them side-by-side. You see that instead of running npm install after checkout, I can just run node install packages, which is obviously exactly the same thing, but it's not. It's doing way more than your regular npm install is doing because this is going to command that's packaged from within the orb and we can look at it, see what it's actually doing.

It's doing way more than your regular npm install is doing because this is going to command that's packaged from within the orb and we can look at it, see what it's actually doing. If you've ever installed any kind of create react app or something like that, you've seen how forever it takes to do the npm install for everything. And imagine if you have to do that every single time you run CI. You probably have to wait a long time for every single commit. That's why you can cache different things. And Node Org has installed packages job which basically does that.

So I'm going onto the CircleCI developer portal and you have this Orbs section here. We basically search for all different Orbs and we're looking at the Node Org again and looking at this install packages command which basically clicking that Show Command Source you can see what it's actually doing and you see that it's doing a bunch of caching to save node modules and stuff like that based on the package.loc that it's generating. So if that doesn't change, you can actually just reuse the exact same state of the packages which is really gonna save you a lot of time and in the long run. And that's like just comes for free with the use of an orb essentially and takes exactly the same or a lot less lines of code. So yeah, that's an orb. Some of them are like very obvious. Some of them can also be entire jobs. For example, we could say node test as a job which would be kind of defined here which would do exactly that. It would install packages, it will run our test but we have our kind of test CI command. So it would kind of be a bit different but you could also do it from an orb. So we have that everything else is exactly the same. Everything else is exactly the same. And then our build Docker again is a lot cleaner now than it was before because we can use like a Docker build and Docker push commands which exactly tell you what each line is doing. And they're exactly like the same otherwise. And our workflow again is pretty standard as it was before as well. We just have build Docker which requires our dependency vulnerability scan and build and test. Dependency vulnerability scan will again use this sneak orb to run a scan for us. So we could see what happens if we do that. Does everyone have a sneak account set up? Cause it's gonna be important, Docker hub and sneak. So, basically I posted the links. Let me just refresh that. But basically sneak gives you a free, I think up to 200 scans per month or something like that. But it gives you a nice free security capabilities. And then yeah, here's, I can't just copy. So yeah, you need a sneak account. I'm not gonna create this one, but I'm just gonna show you where it is. So you log in and what does this do? So here's my project. This is how badly vulnerable it is. Go into kind of your account general settings and you get the API token here. And I will copy it on the other side of the screen. And again, set it as my project environment variable. Let's sneak. So let's leave it up and wait until it loads. Now, we might be able to tell it to execute. This is actually a really, really cool server. The only thing that I would have to RSVP is to just pass on the services. There's a undo button in the back here. And so, I want to find out if I can have the security check, okay, so that was oftentimes the case, then it would be unable to do that. So we don't really want to do that. We might have to look at it a few times for open source projects. So it's our way of giving back to the community. But yeah, you don't have to purchase anything for CircleCI today. Now, if you're going to use it professionally and your teams and that's where you would start looking at signing up for a paid tier. But yeah, no one should be signing up today. I mean, if you want to, I'm not going to stop you, right? Like, go for it. But this workshop, you can run it in the free tier if you're new to CircleCI. Yeah, so I have set up my sneak credentials, so sneak token and I'm just going to show you quickly the stuff that we have going on now. So my build and test job now is actually using the cache to restore what I had node modules from before and still running install, but it's not gonna do anything because it has everything already and then everything is way faster. Can I jump in real quick on that? Sorry, yeah. So what Jean was just showing you was like what I talked about, right? Optimizing your pipeline so that you can make things as quick as possible. Obviously, if you're leveraging cache, the only things that need to be updated are the things that have changed, right? So if you've added a new npm package or you added a new version to the package files, then npm will detect that, write new version, only update those things and then save it back to your cache, right? So this is like an iterative process where you're just updating the things, only the things that you need and not updating everything, which is what, believe it or not, and this is what a lot of people do and it's very slow, very inefficient. So I just wanted to kind of address the cache and how important it is to keeping your pipelines fast and your API token showing, bro. And yeah, I'll have to click revoke and regenerate because I copied it in earlier, but essentially your account, general settings, API token, that's where you get it. I hope you don't use mine. Right, so that's basically it. Going back into our dependency vulnerability scan, this is actually running sneak tests and sneak orb is essentially driving most of that, sneak scan, will tell it to download the CLI and then run the scan. If we had an older version, you could see it actually kinda flagging issues, but I managed to fix all of the vulnerabilities earlier today forgetting that I might have to revert that. So it's not showing anything, but if we had an old kind of vulnerable dependency, it would be flagged here. Um, so yeah, that's like a very quick way to add a scanning, the same for Docker. So we have a step for our, for our Docker build, which is against sneak scan, which again works on the Docker image this time and everything that the Docker image has. So all the underlying dependencies, all of that stuff, it's gonna analyze and I had this failing earlier because it was not kinda, it was always failing on every single issue. But obviously that's like an organizational decision of how, where you set your threshold for having things fail. And, I think you need to go to the different job to scrape. Well no you got it there.

And that's the Snyk job. Let me show you. Actually, yeah, let me break it quickly. Well what Jon is doing here is telling Snyk that if you detect any vulnerabilities in this Docker image, fail my build, right? So that's a way to, what we've done is just to for the demo, we tell it to fail on issues false so that it will accept it and pass it successfully. But that's the conversation that you might need to have with your team, security people, right, to accept the risks of whatever vulnerability is detected. So I guess Jon's gonna show you the broken bit, like how it would behave if it was set to true. So now our build Docker will fail.

While this is failing, I'm a bit wary of time. So I- That actually takes time. Let's go to the third part and final part of the tutorial. Before we do that, is everyone caught up with where we are? To make sure that, because after this third one, we get to do the fun stuff. The more fun stuff. Is everyone not caught up? Yeah, Warren, we're gonna go to the third config. But before we do that, I want to make sure. Wait, we got the environment for sneak config. So there's a link that I posted in Discord, but I also posted above geo. But yeah, as Jean can show you. So it's sneak underscore token. Just pasted it. Can you paste that in Discord? Yeah, and then you just add the value. So actually sneak tells you in an error message that it's expecting that if it fails. Right. Also, again, if anyone is not ready to move on, please let us know in Discord so that I can help you get to where you need to be. But we're running out of time, so Jean needs to jump into the next level. The final stretch. Yeah.

Okay, right. So what I have done is I've copied over this three config final YAML. Um, let me just copy the name into my configure.yaml. Uh, and we'll be looking at that. What this last part teaches us is actual deployment of this application to an environment. In our case, that's gonna be Heroku. And it's gonna test us how to test across. It'll teach us how to test across various multiple node versions, the same projects across multiple node versions. So exactly, for example, node 12, 14 and 15, we can test at the same time with very little modifications. But now let's first go back to our project and see it failed. Okay, so see? Snyk has actually found a bunch of vulnerabilities that we have told it to skip earlier. So that's how it would tell you what's kind of going on. So high severity vulnerability. And these are like Linux dependencies like that are part of Debian that's kinda consists... That basically makes the underlying Docker image that we're using. And yes, this Snyk Docker image search will surface all of that. When I first saw that, I was like, I'm gonna disable that. And then Angel told me, you should just kinda change the flags because it's like very intimidating to see like 400 files scanned and then 700 errors. Yeah. That's what I had and I wasn't pleased, but yeah. Anyway.

So I got, there's a question. So Osama, fixing the vulnerabilities is case by case, right? So it depends on, again, if you are comfortable or you and your team and your organization and security teams are comfortable with the vulnerabilities in your application or in your environments, right? So it's a case by case. It's always different. I worked in the federal government. I worked at in a space program at NASA and the military. So, and I can tell you that because there are vulnerabilities doesn't mean, and they could be high, severe vulnerabilities. That doesn't mean that they're not, they're not going to allow that software to be deployed. The reason is they have other ways to mitigate against those risks. So to answer your question, yes, these could be case by case and in the real world. So, obviously you would like to fix every vulnerability that's out there, but the reality is there's going to be times where you can't because it will break other things. And just kind of destroy the services. So again, it's a case by case and you must consult your teams and ensure that they know that there's a vulnerability and then how it works together to mitigate that. And that's the only way I could really describe it. Awesome. Thank you, Angel. So yeah, the third config that we're looking at adds one orb, which is the Heroku orb, which is a very kind of straightforward tool for deploying a project of any language. In our case, we'll be using that to deploy our Docker image. Yeah, Sneak is pretty cool. Like, it's very easy to integrate it with any kind of GitHub repo, and it'll just kind of scan anything, and especially for JavaScript projects, it's like super easy to add. We still have to build a test job. We still have the dependency vulnerability scan job. We still have the Docker build job. We now have a deploy to Heroku job.

And our workflow is a bit longer than before. Now, let's untangle this. Our workflow now runs build and test in a matrix. Well, that means that bullets fly in slow motion, and it tests things in various node versions. So essentially, if you're thinking about how, like various options you can run each kind of build with, this allows you to specify all these kind of different options, and CircleCI will create you a matrix of various kind of permutations. For example, now we have node versions. It could also have some other dependencies versions, like, say browsers, exactly. Like you could have Chrome, Linux, Mac all that stuff as kind of matrix parameters and run tests across different platforms like that. We're still running dependency vulnerabilities and build docker. This time we're running all three of them in parallel, after which we have a hold for approval job, which will ask us to click whether something is, we want this to proceed, after which we have a deploy to Heroku job. So our graph is gonna look slightly different. Let's see how it's gonna look. So yeah, you can see, okay, I'm building and testing on 15, 14, and 13 running build, dependency vulnerabilities, and now I'm being asked to hold for approval basically. And basically this is like a manual step that you would introduce in an otherwise automated process where you need to actually, a human to go through set of manual checks. You need someone to go through a checklist or whatever, for whatever reason before hitting approve or deny for that matter, and then going on to proceeding with the rest of the pipeline.

Can I jump in real quick, Jean? Yeah. Yeah, so yeah, this is really important and pretty cool, right. So what Jean is showing you here is a couple things, right? Obviously we've introduced, we've streamlined our YAML config to use orbs, which is great because it encapsulates functionality, makes your YAML a lot easier to read, and it gives you the ability to implement things, tools and other services from other partners and the community and CircleCI. One of the things I wanna mention though is, when you're, Dierck can you go back to your dashboard? Yeah. When you're building things like, for testing purposes, you can do like these regression tests on previous builds, which is really, really cool. And you don't have to write a lot of code to do it. It's just literally just saying, hey, run this job in a matrix and then do all of these things. And then finally, we have shown you like the manual process. So I'm sure we all work in places where like, we have a boss or a team, that's like, hey, don't do anything until I look at it. Code reviews, that's where this approved manual approval step is really, really useful. And we see that a lot. At least I see that a lot with teams is like right before code review, before it gets released into maybe a major or a main production branch trunk branch or something like that, this, you know, this workflow process will help with that, right, and enable the people who need to approve things, give them the ability to do that. I generally don't like the approve button, but it's a very useful feature, but I don't like to use them in my pipelines as I like my stuff just to go, you know, automatically, but you have to be very confident as well in the pipelines. And I'll just leave it at that. These are very important kind of issues or features that Jean's showing you how to be more resilient and efficient with your pipelines. Yep, yep, yep, yep, thank you, Angel. Okay, so let's go and untangle these things. First, let's look at how to build things on various platforms at the same time. I mentioned that we're using this matrix keyword which allows us to pass a set of different parameters which can be like anything here. And parameters is another circle CI keyword that is then passed to that job. And matrix basically means for all the different permutations of, or actually combinations of whatever, and this is run that many jobs of that type. Parameters you actually define at the top of your job or within your job. And there are essentially ways to kind of add a bit more dynamism to your builds. For example, I've introduced this thing called node version, which is a type of string with a default value. So I'm passing in a default value so you don't have to kinda always use it. And then in my docker image, which before was just node 15.4, I'm actually passing in whatever that value from parameters is. And the syntax for that is two smaller than and two larger than signs And within those you put in parameters.node version. There is, there is a whole page for these kinda reusable config. Angel, do you mind pasting that page in for that? Because like parameters are like super big topic. I've even made a video about them. Oh yeah, the parameters. You wrote the blog post about them. Yeah, I can post that stuff, sure. No problem. Yeah, that's like a bunch of that. And the rest of the job is exactly the same. It's just like the Docker is kinda picking its own, the docker image, it's picking that value, or version value from these parameters, which are then passed from this matrix of options. If you can think of any other kinda thing that you might want to change in your tests, you can actually do through this matrix. And if you have another parameter, that would be like a lot more jobs run in the same time. We mentioned the hold for approval job, which essentially is a job of any name that has a type of approval. And that will tell CircleCI that's actually an approval job. It doesn't need to be defined elsewhere because it's got a type approval. And to just specify everything else like requires. One more cool thing we have in this approval job is this filters keyword, which basically lets you say, I'm only interested in running this part of the workflow whenever something happens. In our case, that's only on the main branch. Only in the main branch, we want to actually go through this approval and ultimately deploy to Heroku. Sorry. I was going to say Angel Arena, his son, this is actually for Peter, I think he was asking about this earlier. So this is how you would set up filters, right, to say one way you could do it is by in a single workflow, create multiple filters, right, to execute a certain way or execute on a certain condition, which could be a branch name and then you could build different workflows, right, with those filters. So you could have three workflows and maybe you only want one to kick off per branch, right? So if you have a main, which is production, staging, or QA, right, all of those things. So you can filter out and they will only run, you can either say filter, like to execute, yes, or filter, never execute, right, or only execute is what I'm trying to say. So yeah, look up into those things, it should be, look into the filtering processes, and it's a simple Regex, it's nothing too wild. Someone asked if Orb is basically a plugin for CI, CD. It's not, it's a circles, it's like a packaged version of circles CI config, shareable across projects. That's an Orb, so it's just more YAML.

It's just like, instead of you having to write all of that here, all of the sneak steps, they can actually package all their stuff inside of an Orb and you get to use and reuse that very easily. I'm answering Remo. Cool, cool, cool. That's a good question. What are feature benefits of CircleCI versus CodeCommit, CodeDeploy from AWS? Can we jump to that one after? Because we need to... Well, we'll finish this exercise and then we'll give you an exercise, the one for the prizes, so yeah, ultimate step is essentially this Heroku deployment, which again, requires a HerokuAppName to be set and I think we need HerokuToken, which we can set now. HerokuAppName, we'll use this CircleSampleNodeDev. Which I already have created, and that would be for your own project and again, the same for the rest. But this is the last step we'll do, so you don't actually need to go on and build the Heroku step. Actually, let me just quickly do that, let me just quickly do that so that you see it going. And then we need a Heroku token, I think it's in the orb. So I think now you have everything that you might need for our little assignment. So let's go check authentication, Heroku API key, that's the one we need. And we're gonna add that and I'm gonna go to my Heroku and copy my API key. Well, I was answering that. If anyone is having issues still, I know I see like Remo or any, you know, anybody in discord, just flag me in discord, let me know so I can help you out. And we'll have plenty of time for everyone to, the ideas for everyone to be able to complete the prize assignment. Because I want everyone to have a chance to win some stuff. Trying to give this stuff away. Absolutely. Cool, okay so I've added my Heroku environment variables. Now I can hopefully rerun this from start and see it pass. And yeah obviously I'm running everything from the main job so you don't see this kinda stop before hold for approval. If I ran this on a feature branch for example, this would not even get to this hold for approval because we have that filter setup. Anyway, let's talk about prizes. So we have an assignment for you. Oh hold on before you continue. There's a, Peter has a question. Peter is that what environment variable is that for Heroku or for? Yeah Heroku. Yeah it's for Heroku. It's not, let me go back to it. It's not a requirement for the rest of the exercise. I just wanted to show it to you that it works. Oh, it's not, so this is not required for the price, right? Yeah, you don't need it for the price. Okay, good that's good to know. Yeah so but it's Heroku API key, which is what you get in Heroku dashboard and Heroku app name, which is an app you need to have set up in Heroku. Anyway, but we have a price going on. I will update this link with a new API, but essentially I want you, well we want you to hit an API end point to create a job from CircleCI that hits an API endpoint that will give you with a similar payload. So CircleCI-build-url, only in JSON with a URL. And we alluded to where you can find this earlier in the beginning, this is essentially set as an environment variable in the build, in the jobs you have. So every single job that runs will have its circle-build-url which will link to that specific job. For example, that exact job. So yeah, tldr, create a job, make CircleCI, hit an endpoint, I will be refreshing that URL shortly and giving it to you, it should be a POST request and the payload should look like this. Essentially, it needs to contain circleci-build-url and we'll verify that you have done it by actually going to that link and seeing your job in action, actually. Let me now Let me now make sure that this runs. Yeah. Yeah, wanted to give everybody ample time, so yeah, add the job and then we'll help you if you need any help getting it done. Because we're just gonna gather all of the entries and then we're gonna randomly select. You'll see, we have a service that we're gonna show you. We'll dump everything in there and then we'll pick a winner. Is it something to do now or like later? Yes, yes, this is part of this exercise. Jean, can you just post all of that into Discord? Absolutely, absolutely. In there, yeah that way they... Yeah, that way they have the... there you go. Yeah, so you could actually tack this on, right? Yeah, so circle underscore build underscore URL is the environment variable that will be set from within any kind of circle CI built environments from a job so that you can run it from there. Whatever language you're doing, I've implemented this in cURL earlier. I would imagine it's probably easier to create it from as a simple node script and just run that NPM, NPM that. But yeah, that's what we want you to do. And everyone who completes this in the next 30 minutes will get circle CI swag and will draw how many prizes do we get. It's literally just a job that sends a post with that value. You need to pass in that value circle CI built URL. Obviously it needs to be JSON formatted. It doesn't have to be on the existing config but it will help if it is. Maybe show them the curl. Just show that. If they don't want something, some people are not programmers. I will show you the curl from a Postman collection that I have, which is here, I think. Can you see that, no. And I broke it, anyway.

Let me do it, let me do it from here. Copy, post, body, raw. Why don't you zoom in a little bit. So that thing is very awesome. Yeah, so again, just post that payload from a job. If you have a script you can run. If not, just use this curl command that he's gonna share with everyone in the moment. You should, Kmesh, if it works, you'll get an OK200, right, off of the API. I hope. Get 200. I don't think there is a negative value but we are collecting all of them. So yeah, that's the response you should, that's a request you should send. I'll just print out the curl. Yeah. Post that in Discord I think too, if you would please. That would be great. There you go. So build a job that writes this or executes this in a step? Executes the step. Yeah. Yeah. And if you're having issues just raise your hand, let us know. Is it okay to add a new step to build and test? It's wherever you can do it, you can create a brand new job, you can create adjust the new step, it doesn't really matter. No, you don't have. Yeah. Yeah. Oh, wait yes. So he's asked, someone's asking about the build URL. Yeah, so just grab... Yeah, you're asking for there. No, no. No, you don't have to add your URL you just have to post. Yeah well, it should have your own, it should come from your job which generates you this build URL, yeah. Your curl request is wrong though. You gotta fix that, edit it, put the variable, remember? Yeah, yeah. Yeah, fix that curl. Kay Masher's right. Yeah. So Jean, can you replace that curl request or edit it? You have to, in the payload, you have to make it the CIRCL CI. Oh, right, sorry, yeah, absolutely. That's why they're asking. Absolutely. I have it written somewhere. It's right in the Discord, in the values in the Discord there. You already posted it. Put it in the previous one? Yeah, yeah, yeah, I will. I'm trying to give you an executable snippet. Executable snippet... Yeah. Because Bash is hard. And Bash is hard. And I will open this, Yeah, yeah, that's the one we want. Obviously the URL is gonna be different. The URL is going to be different, it's gonna be this one. Yeah, someone's saying, asking, we can send a lot of build numbers. You can but we're going to filter it unique so, doesn't matter. You're only gonna get one entry. We thought of that. We're sneaky too. But thanks for calling it out. I appreciate it. We're all about the hacks. There is no warp. The main part of the assignment is you get to do something with CircleCI that hits our API. It hits our API endpoint with your CircleBuild URL. Whether you create a new job or you add a step to an existing job, it's all up to you. But you should have everything you need from this workshop. Ultimately, you just need to make it run this Curl command from CircleCI. Why don't you show your example? I think it would be fair to... We've shown them a lot of things today. That's true.

So, we're trying to make you all think, but this is something, a way you could do it, right? Highlight it for them, so they can see. This is a brand new, empty project. Empty, empty branch. Empty config that just has one job, hacketyhack, that runs this command. And the workflow just runs hacketyhack. You could add that block of hacketyhack into whatever step in your job. Another job. All you care about is that run command. That bit right there. And make sure it's inside of a job that's in one of your workflows. And that's it. Does everybody understand? If not, let me know. Let us know. I want everyone to get a fair shot at winning. I think I broke something. I think I might have broken something. It's not receiving. Hey, at least everyone gets to see us debug live. Great. Let me try this again. File is empty. It contains one job. A little bit less. Yep. Yeah, actually run it because I had it broken. And I'm updating the URL now. So, this should populate. Oh, sorry, I messed up again. My snippet is wrong. It should be slash. Well that would help. All right. Look at that. We got one. It's working. So yeah, we got five of these books and one pair of headphones to give away. Hey, check your thing again. Someone's complaining there's nothing. Yeah, yeah, yeah, darkness. You're not using the right. Yeah, make sure that everyone update to this new Ngrok value. It needs to go to this F970755, whatever. The endpoint changed, so we need to. Yeah, because my thing crashed. You're getting 404s. There was a GET. Oh, okay. There you go. There was a GET, so you need to do a POST request. If someone's posting, whoever is posting without the route... Slash pipeline slash register, you need to do that. There you go. Oh, wait. What the... How does this...? This makes no sense. I think we just dumped the names out of the participants. Got some... Help me, I love it. Did my app crash? Yep. Oh. You got no error handling. Only we had tests. Yeah. Let's go ahead and just dump everybody's name that was left. The ones that hung in there with us. Yeah. That's how we're going to do this. Yeah. We will make this better in a future workshop. But, yeah everyone, everyone is eligible for the thing.

What... Oh, wait. Yeah, I think we're running out of time, man. Yeah, we're running out of time. Let's just dump the names that are here into the system. Yeah. Do we get those names from Zoom? Looking at the participants list. I don't know how to export. Let me see if I can export the names.

Okay. You know, I'll just... Do you want me to run? I can import and enter them. We'll just have to manually enter them. I don't think we can export. Yeah, we'll do it. And we will do them in... What's that app you said? Pinwheel.com? Prize wheel? Prize wheel. No, that's not it. Where is that app? Picker-wheel. Picker-wheel. So we'll put the names in here. Do you want to do it on your desktop or do you want me to do it? I'll do it on mine. Okay. We'll put the names of the folks in the workshop. We're going to use the name, the participant names, in the participants list. So if you want to shout at this. There's a couple of names there, they're all pretty legit. We can tie you back to your account later. But we're also at Discord, so this is a good time to get in Discord and sign up.

It was a noble idea, but we— Yeah. This is— Oh, sorry. Go ahead. Did I mess up something? You talking to me? Yes. You said, up something. No, I don't think you did. Okay. I'm trying to find your— Okay, there you are. Okay, and we will be—if this goes public, obviously we will cut that prize part with the names from the recording, so don't worry, it's not gonna enter actual. Okay, I'm not letting anybody else in. I think Alexei, this is Alexei, are you the same Alexei that I just let in? I was assuming that was you. Okay. Don't worry Lobster, if you're on the Zoom, you're gonna be entered—we don't know your real name, but you can get in touch with us, DM. Peter, yes, you're in. Okay, let's do this.

First one is for the first of the five books. Riccardo Ollando Riccardo. Make sure you check it off, yeah. Congratulations. You want to win? You want a book? Can you ping either Angel or myself? Dorian, congratulations. You got a book. I need to give you a plus one. I don't know how to do a plus one. Anyway, Dorian, you won a book. Thank you. MZlong, you won a book. Alejandro, are you not in? Did I miss you? You are in a chat. That's weird. I thought everyone who was in Zoom... You are in. OK, good. He's there. This is how many books now? Are we keeping track? We had 3 books so far. Fourth book. So you're on the fourth book. Alexi, Maiko. All right. 5th book, congratulations. 5th book, last book. Mark, congratulations. You won the last of the books.

Oh, yeah, absolutely. So we've got... M. Ziellon, Alexis, Dorian, Ricardo and Mark. Okay, now for the grand prize, headphones. Peter, you're still in. You ready? Yeah, drum roll. Pradeep, congratulations. Congratulations, Pradeep. Congratulations. Yes, I'm afraid that's all for the prizes. If the people who won would kindly ping us on Discord with their details, because for these we actually need shipping details. Actually, there's a question though. The Swag for All, we have all your emails. Do we? Well, in Zoom they've signed up with email. That's right. Okay, so if we can match – if you don't think we can match your details with an e-mail, please let us know. So if you signed up with a fake e-mail, if you want swag or a prize, you need to… Yeah, if it's like a nickname e-mail or something like that. And your name is there, but not in your e-mail, please let us know, ping either myself or Angel and we'll make sure to send you swag. Yeah. All right. Thank you everyone. Well, I think we have some questions from before that we never answered. Yeah. For participating anyway. Yeah. So please send us… If you have an alias e-mail, whatever, we need to get in touch with you somehow. But I think there was a question from Mark C that I wanted to... Yeah, Mark C. Mark C, I'm sorry. What are the features slash benefits of CircleCI versus using CodeCommit or CodeDeploy from AWS? I have not used any of those products, CodeCommit or CodeDeploy. Basically, CircleCI is a platform company, a product company that does specifically specialize in CI, CD. That's all we do. That's all we think about 24-7. Every single engineer is working on that. AWS CodeDeploy and CodeCommit are one or two features or two products of the AWS ecosystem. So, they're obviously a lot less focused as an organization to that. That's one thing. With CircleCI, you get obviously all the 100% focus and with AWS, you probably don't get that. Are there benefits of putting CircleCI on a VPC and use it there versus using CodeCommit or CodeDeploy? Angel, you want to do that one? Yes. I've actually used all of those things. I will say that anyone who's ever used CodeDeploy from AWS, it's a hard one. It's okay. There's two things going on. So CodeDeploy is very YAML-centric, right, and you have to understand how AWS works, right, within itself. And you know you have to stand up – or you don't have to stand up, but they process your pipelines for you. And they're not the best things to look at and best things to manage. With CircleCI, we also use YAML, but the beauty of the platform is to enable developers to easily implement CI, CD, without having to manage all of those things, like Compute and all that, right? So when you're comparing us to Code Deploy, we are solely focused on enabling developers to implement continuous delivery as easily as possible. That's why we zero in on developer experience. Even though DevOps, continuous delivery, these are all concepts that are primarily focused on more of the operational side. But I will say that with CircleCI, right, we are solely focused on preventing complication, in a sense, right? So when you use other people's or other companies' CI tools, they have, they only think of CI as like a feature, right? So they're trying to sell you another, like AWS is more of a cloud provider, but they don't have, in my opinion, very good orchestration tools or they're very complicated. I don't say very good, but they're complicated to use. And that's where we come in as a different angle, right? It's just making it a lot easier and giving you as much capabilities as possible. You could deploy CircleCI by the way, it comes in what we call a server flavor, enterprise type flavor, where you could actually stand all of this up, right, CircleCI up in your own on-premises or in your own cloud even. But, you know, for the average developer and team of developers, they don't want to manage infrastructure, right? They just want to fire off their job, using a git push, and then get some sort of feedback loop in five or ten minutes or less than that, right? So that's kind of the difference is we're all about making it super, super simple and easy for people to implement and understand versus some of the other ways people do it, right? Which I'm not saying it's wrong or right, it's just we're more focused on the developer and the experience where the other tools are more cumbersome, more comprehensive, and they require a lot of knowledge about the specific platform you're using. And another thing is, you know, where... when you're using those other tools, you're gonna be locked into their, you know, that like the code commit specifically only works for Amazon, right? With CircleCI, you can attach the different services, right? And it'll apply to any kind of, you know, service, right? So we're pretty much agnostic as far as the technologies that you could use. We're pretty much open, and that's the whole point as well, right? Is to give as many capabilities as possible. So I hope that answered your question. It's kind of long. But for the most part, that's where we are. We're a very focused CICD. That's all we do, right? It's continuous integration and continuous deployment. That's what we're focusing on. That's what we're really good at. We're not trying to do all-in-one kind of things. Yeah. I hope that answers the question. Thank you. I appreciate it.

All right. So, yeah. Someone asked, who are the winners? You are all winners. We're going to get some swag. The ones that stayed behind here, we have your names. And then If you won something, we've obviously told your name during the stream. Then please reach out to us. One person already reached out to me with your address because we need to know who to ship it to and where to. So... Yeah. DM me or Angel or tweet us or email us. However you can reach out to us, please do. We are in this Discord chat, so it's probably the most convenient. Yeah, we'll be monitoring this Discord chat for a couple days. So, like... Don't feel bad or don't feel like you won't be heard. We'll be... Your cap is already... Haha. Yeah. This is an original. No, so yeah. We'll be monitoring this but, yeah, thank you everyone for showing up. I really appreciate any feedback. You know, how we can do better. Obviously, we'll build the application to handle some errors. But, at the end of the day, it was a lot of fun for me. I'll let you close it out and thank you everyone. Look forward to speaking with you all. Oh, and I'm also giving a talk later on... No. Is it next week? Yeah, next week at the conference. So, if you wanna jump in and learn about increasing your app confidence using CICD, I'll show you how to build some Kubernetes stuff from pipelines. Yeah, that's gonna be cool. I'll share this. I'll clean this slides up and share them, and, obviously, all the code is available on the repo, and hopefully, you have followed along. So, if you have any questions, it's been a great pleasure taking you on a tour of CircleCI and breaking some builds with you in the process. That's always fun. Yeah. Thanks again for everyone. What about the swag for all T-shirt and stickers? I guess you have our details. We have your names, and I believe we have the emails. If you don't think we have them, please do ping us on Discord or Twitter or wherever. My email is zan at circleci.com. I will drop it here. Yeah, we're only going to send to 26 of you who stood by us. That's the people who get the swag. Obviously, the winners should reach out to us again because we need your shipping addresses if you want the prizes anyway. Thanks again for attending. Congratulations. Thank you. Thanks again for attending. Congratulations to the winners. I hope you have a wonderful rest of the week. And Angel plays something with his guitar. You want some outro music now, too? C'mon, guys! I tell you what, Peter. Next time. We'll play a little Pearl Jam for you.