Enter CDK: What this Means for the World of IaC

Hey everyone, I'm Roy and today I'm going to talk to you about aws cdk and what does it mean for the world of infrastructure as code using aws cdk. So today I'm an engineering team leader at Firefly and what we do at Firefly is to take your entire cloud footprint along with your kubernetes clusters infrastructure and on the other hand to take your infrastructure as code, if it's cdk or even Terraform, Pulumi, CloudFormation and even Helm, and to combine them and to show you a merged inventory that shows your entire cloud asset footprint in the means of your infrastructure as code. So also as you can see I like Schnitzels, Call of Duty and infrastructure as code, I guess the same as all of you. What is actually cdk and what is the huge hype about cdk? So when I was visiting the last aws event I couldn't stop noticing that everyone is asking about cdk. They asked how to start working with cdk and in what languages. They wanted to know how they migrate their old CloudFormation stacks into cdk stacks and I think that with all this question I started investigating as part of what we do at Firefly which is to use the best infrastructure as code and to supply our customers the best infrastructure as code and after a quick run over cdk I discovered an amazing new tool by aws that you must be familiar with and an even easier way to migrate your CloudFormation into cdk. So what is actually cdk? cdk aka cloud Development Kit is a concept not only by Amazon also by kubernetes and also by Terraform themselves. cdk means that you're not using any config language to create your infrastructure as code such as JSONs, HCLs and even YAMLs but you're using coding languages. In aws cdk they only now support typescript of course and javascript. typescript by the way is the language that it was written in so it works best in typescript but also javascript, Python and even Golang. Besides cdk is the CloudFormation successor. cdk is the brand new fully oriented aws tool to create resources with and it's much better than CloudFormation in let's say three major points. The first one is that CloudFormation is messy. If you're going to write lots of resources because of its super explicit way of approach even CloudFormation is going to be a super huge YAML or JSON depends on your benefit but if you have much resources it's going to be a heavy file and not so likeable to work with. Also if you're going to create relations in your CloudFormation it's going to be a hell because it's making it super messy, super buggy and not even understandable for anyone that doesn't know it. cdk solves all of these notes because cdk uses classes and variables such as any other coding language to create the resources with it. Also another major point is the multi-regional way of working with CloudFormation. As you already know CloudFormation does not support multi-regional and if you want to deploy your CloudFormation stack in multiple regions you'll need to use a stack set. A kind of a patch that aws wrote to deploy one CloudFormation stack in multiple regions with cdk it's basically solved because cdk supports the stack out of the code. You can create as much stacks as you can inside your code and just deploy them one by one into each region you wish. Also cdk is basically now the native experience of aws which means that it's not only 100% aws-oriented. It's the only way to, the only thing you can write with cdk is basically aws so it's not fit to any other providers. It's full aws service so it means it works best with aws and only aws. Even more than this as I already said cdk supports multiple languages. The most typical one is typescript but you can write also javascript and much more languages with it which is amazing because basically if you think about it you can write one repo, put in this repo your already existing coding or programming functions. On the other end you can put in the same repo your infrastructure as code such as in cdk and then if you let's say deploy an instance with some code you can define the instance itself in the same repo which is basically a branding amazing way to work with infrastructure as code. So how does cdk work? cdk turns code into CloudFormation stacks which basically means that aws wrote cdk on top of the old CloudFormation mechanism. You're going to write it in each language you wish as we already described and you're going to write constructs which means like creating an SQS resource or creating an S3 bucket resource or an instance resource for example. After writing these things in these resources you're going to use the command CDKsynth which is basically a dry run of what you already wrote and it will generate automatically the CloudFormation template that describes exactly the resources you wrote in any other coding language you used. So creating the CDKsynth created a template and then following that you're going to use cdk deploy to deploy the stacks by actually creating CloudFormation stacks with this template inside and creating out of the stacks the resources on your aws specified region or regions for example. Deploying will create two artifacts. One an S3 bucket file that run the state let's say of your cdk code. If it saves the resources that were created the changes that it made and you can track over there and see the state of the lately deployed stack. Besides it's going to create the CloudFormation stack itself which creates the resources out of it into your cloud. So who should use cdk and when? It's a pretty typical question because the who is basically everyone. Everyone that uses aws because cdk is the 100% oriental way to approach let's say to create infrastructure as code over Amazon and especially CloudFormation users that wish to migrate the CloudFormation old stacks into cdk. It's going to be a pretty easy process to go through and we'll do it in the example following that. Another group is like any devops or SRE teams that are working with aws and besides if you can think about it even your developers can use cdk because now it's not like a knowledge you need to experience with YAML or config. It's just coding the same as any other programmatic coding you do. And besides all of the infrastructure code users who work over aws it's best to work with cdk. When? When is I think every time because if you're already using any infrastructure as code it's always a good way and always a good time to start working with a tool that is officially working with aws. And besides if you're a CloudFormation user then you needed to do it yesterday basically because cdk will let's say spare you lots of suffering and hardship working with CloudFormation. The pros and cons about working with cdk I think the pros are pretty straightforward and we already talked about them. It's the support of multiple languages. It doesn't require any config language, the multi-regional benefit and the aws orientation that is basically building. Who should not use cdk? Not using cdk is in my opinion when you only use instead of working with only aws as your provider you use multiple providers. Let's say Datadog and Auth0. If you're working with more than one provider as we said because cdk is the aws tool you need to work with another infrastructure as code language. And at this point it gets a bit messy. So if you're working with aws I think the best approach is to cdk but besides you should think about it and maybe use one of the other solutions for working with infrastructure as code instead maybe Terraform or Pulumi or any third party that doesn't stick into one cloud specifically. At this part I'm going to show you a bit about how to migrate your CloudFormation into cdk and besides how to create resources with your new cdk stack. So as you can see here I demonstrate a pretty basic stack that creates a CloudFormation cdk resources and alongside includes an old CloudFormation stack that I already deployed on aws before and now I want to migrate it into my cdk project. The stack as you can see contains an instance, a basic instance that doesn't do too much but already deployed on my aws account and at this point I will need to migrate it into my cdk alongside I created an S3 bucket and an SQS queue that all of them are newly resources that created inside my cdk stack. By running the basic command cdk synth I'm basically planning this into my cdk stack and into my S3 bucket and creating a plan of the dry run of what we're going to deploy soon. After the plan finished I can just run cdk deploy and in a heartbeat these three resources alongside the already migrated CloudFormation stack are going to be deployed through a new CloudFormation stack that is described by my cdk stack in this code. You can see it in my aws account that when I create a stack it has three resources inside the example instance that was migrated for the CFT stack that I imported and also the bucket I configured and the queue I created newly. After doing this thing I basically can just create another includes as much as I can and also create new resources alongside and I have here my first typescript cdk project. So this is basically the entire cdk hello world and I think that the two key takeaways you should consider after this talk is the first one cdk is a rising and amazing technology that will probably be the best tool to manage infrastructure as code over aws soon, if not even now, and you should consider working with it. The second one is that each day you're using CloudFormation is a waste and you should totally consider working with cdk and as you can see the importing of all CloudFormation is super easy so I think you should basically work with cdk at this point. Thank you all I was Roy and this is my mail feel free to contact me with each question you have regarding if it's YAC, infrastructure as code or aws management and let's keep in touch. Thanks. Hey Sharona. Maybe a little bit more surprising than I thought I mean I knew that Terraform would be the large majority but 76% and 0% a big zero on cdk so it's a it's very very new I'm guessing. Yeah well it's not even surprising because I knew that Terraform is going to be the major here and the zero percentage of cdk just showing us how much cdk is new and it's not still being fully fulfilled of its potential. Yeah it's not well adopted yet but what's actually interesting is take a look at that number two is other so I wonder what that is like homegrown tools folks feel free to drop in the chat in the discord what the other is let's find out what other tools you're using besides Terraform and say like the other ones that are on the list and that aws CloudFormation is more popular than Pulumi actually did surprise me as well. I felt like people had moved on from CloudFormation once Terraform and Pulumi were around so that's interesting. All right cool those are our results but I'm waiting to hear what folks have to say about their other oh here I see Pulumi is rising now so okay I take it back and other has gone down a little bit but I really don't know what that other is it's very intriguing to me. Yeah so let's take a second and see what the community has to ask and what they want to know so first and foremost I guess one thing that's really interesting about you know aws and cdk in the context of you know infrastructure in general is how does it handle things like drift detection that maybe Terraform is a little bit more well equipped and is known to have like kind of guardrails for so what are your thoughts on that? Well it's a pretty good question because as cdk successor of the CloudFormation is dealing it the same. The CloudFormation way to deal with drifts is to show on all the supported aws assets the complete drift that would happen if you're going to run a stack deploy or stack plan. Can you hear me sorry and in this case if you're going to run any stack deploy or stack plan you can see the drift through the CloudFormation and cdk acts the same way exactly. It supports all of the aws supported assets and it shows the drift completely. It can help you analyze the drift in the inner property of the assets which is a really cool feature Terraform doesn't support it yet another great advantage to the cdk and through this feature you can basically monitor drifts and see the current state of your infrastructure code regarding the resources that it was created. So it's a really cool and nice feature you have with cdk another great advantage here. Awesome that's cool I was actually thinking about in the context of Terraform but you actually enlightened me I didn't know the CloudFormation has that kind of drift detection as well I've only really heard about it in my kind of Terraform context but I'd also if you want to kind of allude to a little bit about Terraform Pulumi in that context as well I'd love to hear like how they handle it just interesting to me. Of course then Terraform has also a way to show drifts it's a pretty brand Terraform plugin that they developed Pulumi as well supports it because their Pulumi as we said is based on Terraform and it's also showing the drifts and you can basically take the Terraform output to fix them it's not going to do it automatically as aws cdk does the biggest advantage and it's also not going to be that specific as a cdk can get into if it's like showing the CIDRs of a security group if it's getting into policies and roles and it can open the JSON and show the difference completely and it can help you fix it and Pulumi and Terraform do support drifts but not that good as a cdk of course because it's originally been created by aws so. Yeah that makes a lot of sense well you've got folks intrigued that's for sure I have a question coming in from the crowd CC Miller asks well got me interested in cdk and he wants to understand how complete newbie friendly is it really he says he teaches a 16 week intensive boot camp for full step javascript called school of code but he wants to know even for him how easily adoptable it is. Well as you saw in the demo over there and I think it's pretty adoptable and let's call it yeah it's newbie friendly and the usage of cdk is pretty straightforward you can basically pick any this was a demonstration in typescript but you can basically pick any coding language you can think of and getting into it even if you already have a project in your needed language is going to be super straightforward you can just start by commanding the cdk in it which basically creates the infrastructure that it needs the supported infrastructure on the aws console side and then you can just start writing resources and cdk sync and deploy will in a second in a heartbeat will deploy them into the cloud I guarantee that in the hello world example is going to take no longer than five minutes and if you're going to start migrating a cloud formation for example or getting into a bit harder resources it's also going to be a super super easy way to go in. That's cool. Julie Julie chimes in with saying she thinks that the other is probably azure arm or bicep but I wonder what anybody else is thinking if you are using azure resources or other maybe Google cloud feel free to chime in and let us know we really really want to know what that other is don't keep us in suspense. More questions so in which way does aws cdk prevent the usage of aws stack set for example. Of course so that's a good question actually and stack set is basically Amazon's answer to the usage of cloud formation if you need to use it in a region way and stack set is a patch let's call it that was built over Amazon cloud formation and you can use stack set to deploy the same cloud formation stack in different regions as much as you want but it's also say meaning that you need to control another resource and to support another resource and it's going to price it has a price and it says maintenance value. With aws cdk you're basically creating the stack let's call it or the component that is going to be deployed in your code by doing that you can create as much stacks as you wish you can create them in the same project or create them in different projects and by that you can basically take the same component or the same bunch of resources you can unify them as a simple stack and then you can deploy the stack in several regions by doing it through the code so it's a great advantage doing it is a super super easy way and yeah with cdk of course and by that you can basically prevent usage of stack set or cloud formation specifically you can just migrate your cloud formation into your cdk and with cdk deploy it in several regions and then prevent the usage of the stack set which is basically the right way to go this way these days. Oh cool I mean I was gonna ask but you kind of maybe covered it like how complex is that what's the complexity with actually with running it in multi-region is it like so it's not so not so hard you're saying. Super super straightforward yeah super straightforward you just gonna need to create several components and give each other different regions to go with and connect them to the component you wish to deploy. That's cool awesome it sounds really cool also I'd love to ask the folks after this talk how likely are you to check out cdk and see how it runs and try and make your own examples I think this was actually a very enlightening talk I heard that it was like all the buzz that aws reinvent cdk everyone was talking about it. You were there I'd love to hear. Yeah so part of the reinvent was like at Firefly most of what we do is to handle infrastructure as code and by that and when people stopped at our booth and started asking questions most of the questions were related to cdk and when are we going to support it and how deep are we going to support it and by that after we started to investigate about aws cdk when we saw the huge benefits of using it and since then I keep hearing about when the aws cdk and the like people trying to migrate into it from cloud formation from Terraform even which is like crazy and we keep getting question about how to do it well and how to migrate well and what's the basically the complexity as we talked before doing that. Yeah I mean I've been seeing a lot of like Terraform for cdk, CDK8 projects like cdk for kubernetes what's their like kind of maturity what do you know about these projects if anything? So basically the concept of cdk as we talked before it's pretty new and the K9 cdk, the kubernetes cdk and the Terraform cdk are a pretty new project they're not still developed like completely you can use them and I know about companies that do use them but the aws cdk on that point is much more mature and much more suitable for usage. It also took some time but it started a few months earlier and with the aws great coverage and just basically the creating it over the cloud formation which is already a prepped up technology just made it much faster and created the maturity of it much more higher. It sounds almost like it's like competitive or a contradictory Terraform for cdk how does that work like I'm trying to understand how like Terraform for cdk works. So it's not that of a competition between them because always there's the big huge difference about the multiple providers Terraform can hold. aws cdk is like basically created for aws in the manners of using it over aws there's no doubt that the aws cdk is the right choice with the originality and the way deeper understanding of aws resources and on the other end if you're going to use like several providers and you need the versatility then the Terraform cdk is a great choice for you. Still as I said not much matured and I still see a lot of pros about using the Terraform and the classic Terraform and not the cdk Terraform but it's already on the way and I guess in the next month we're going to hear much more about it. Awesome yeah this is on a personal level aren't you happy I encourage you to come submit this talk and represent our community. I can say I'm proud of you and I think you did a really good job so yeah that's a really great job. I actually know there are quite a few of these but I wonder what you are familiar with and what you know about them. So is there such a thing as a graphical creator for cloud computing like drag and drop network design using a low code solution or some kind of easy config UI based tooling? So I know the answer of Amazon to this concept which basically can be shown as a step function and one of Amazon's latest tools it's not a new tool but it's got super hype lately it's step function basically the concept of creating your own flow with Amazon resources you can select like tons of resources out there create a flow and connect them with events and with the triggering and every technology you can think of you do it graphically as you describe like you have a canvas and you just drag and drop resources and connections between them you describe the connections and you describe the resources themselves and you just create your own flow on aws. So I'm super familiar with this concept over aws I know that other clouds and even third parties support it also but I don't go much into it lately. Okay that's a great answer thank you so much for your time Roy thank you so much for joining us and for your excellent talk. Thanks everyone.