2. Building Real-Time Serverless GraphQL APIs
I'm going to be talking about building real-time serverless GraphQL APIs with TypeScript, AppSync, and CDK. We'll start with an introduction to GraphQL, then discuss AWS AppSync and Manage GraphQL Service, followed by a brief intro to CDK. I'll also do a live coding demo to demonstrate how all these components work together. GraphQL allows you to define a schema that represents your data types and operations. It offers more control over API latency and data transfer, returning responses in JSON. A GraphQL API consists of three main parts, starting with the schema.
Okay, hello everyone and thank you so much for attending my talk today. I'm really excited to be talking about this subject because it's something that I've been really using a lot lately and it's really powerful. So, I'm going to be talking about building real-time serverless GraphQL APIs with TypeScript, AppSync, and CDK. My name is Nader Dabit, I'm a Senior Developer Advocate and I'm by trade a front-end and mobile developer. And I've just now been working with AWS and back-end services for the last couple of years. And lately, I've really been focusing on building full-stack cloud and full-stack serverless apps. So, let's go ahead and jump right in because we have a lot to cover.
I'm going to be talking about a couple of things and they all kind of fit together. So, we're going to first start off with an introduction to GraphQL. We're going to then talk about AWS AppSync and Manage GraphQL Service. We're going to then do a brief intro to CDK. And then, we're going to kind of talk about how all of this stuff fits together, AppSync, CDK, and GraphQL. And then, I'm going to do a live coding demo. And I think the live coding demo kind of will show you how all this stuff works together better than any presentation will. But hopefully, putting all this together will be really helpful for you and you'll learn something.
So, let's talk about GraphQL for a bit. GraphQL, if you look it up on the internet and you find the definition, you might see something like this. A query language for your API. This doesn't say a whole lot. Let's dive a little bit deeper. With GraphQL, you have your, almost like a menu of your data, which is your GraphQL schema. The schema has all of the different data types, which are basically types, and all of the operations against those types. So, you have maybe a to-do app. You might think of something like a to-do type. And then, you might have operations for creating, reading, updating, and deleting your to-dos. All that is defined in your schema. And, once you've deployed your API, and we're going to talk about that a little bit more in just a moment, you can then just ask for the data that you want. And this is kind of the thing you've probably heard about before, if you've ever looked into GraphQL, is, one of the really powerful things about it is, you have the ability to build much lower latency and more, I guess, controlled APIs than when you're working with REST endpoints because you have a lot more control about the data that is getting sent back and forth across the wire, and then you get your response data returned to you in JSON.
So a GraphQL API is made up of three main parts. You have your schema, which we talk about in just a second there.
3. Understanding GraphQL Schema and Operations
We'll dive into a little bit more in just a second. You have your resolvers, and then you have your data sources. So, the schema is where all of your data declarations live. So, again, for a to-do app, you might have a to-do type, and then to operate against that to-do type, you might need to define some operations.
We'll dive into a little bit more in just a second. You have your resolvers, and then you have your data sources. So, the schema is where all of your data declarations live. So, again, for a to-do app, you might have a to-do type, and then to operate against that to-do type, you might need to define some operations. So here, you might have a way to get a single to-do by ID. You might have a way to create a new to-do using a create to-do operation. And these are defined in queries, mutations, and then we also have a new type that is kind of unique to GraphQL called a subscription, and we'll talk about all three of those in just a second.
4. Understanding GraphQL Data Sources and Resolvers
Once you've defined your schema, you have your data sources. The data sources are where the data is coming from for these operations, and you'll get the data returned to you in one of the types defined in your schema. Operations are handled in a resolver, which maps the GraphQL operation to a data source. This is an implementation detail for you as a developer, and frameworks or services like AppSync may have their own opinions.
So, once you've defined your schema, you then have your data sources. And the data sources are where the data is coming from for these operations, and then you're going to get the data returned to you in one of the types that you've defined in your schema. And the way that those operations are handled are done usually in a resolver. And the resolver are really essentially just a function or some type of code that maps the GraphQL operation to a data source. And this is an implementation detail for you as a developer if you're building out your own API, this is going to be kind of up to you how you write this. A lot of the frameworks have a little bit more opinionated ways about doing this. And if you're working with a GraphQL as a service, something like AppSync, those are also opinionated as well.
5. Data Sources and GraphQL Subscriptions
So, a data source can be a database, a function outside the main service, or an HTTP endpoint for microservice architecture. GraphQL queries, mutations, and subscriptions map to HTTP verbs. You can request specific fields in a query and have more control over data access with GraphQL as a data source. GraphQL subscriptions enable real-time communication between a client and an API. Subscriptions are event-based and typically have names like onCreate, onUpdate, or onDelete. Implementing GraphQL subscriptions can be done using services like Hasura or AppSync, or by making decisions between servers and events or websockets.
So, a data source can be either a database, a function that is kind of outside of the main service itself, or an HTTP endpoint for bringing in a microservice architecture into a GraphQL API.
So, let's talk about the data fetching piece of GraphQL, which is really interesting. With GraphQL, you might be coming at this from Rust, so I think it's really convenient and really interesting to kind of compare the two, GraphQL versus Rust. With GraphQL, you have this idea of queries, mutations and subscriptions, and these map really well to things that we've done in the Rust world, kind of mapping to things like the HTTP verbs that we work with. So, when you need to read some data like getting or listing a list of items, you're going to be using a GraphQL query. If you're going to be updating, creating or deleting anything, you're going to be typically using a mutation, and then if you need some type of Realtime aspect to your API, you're going to be using subscriptions.
So, let's take a look at a table of GraphQL versus Rust verbs. So, again, mapping a Get request might make a lot of sense to a GraphQL query. So, getting an item by ID or getting a list of items will be typically a GraphQL query. And then post, put, delete or patch operations will be GraphQL mutations. So, that being said, let's take a look at kind of the way that GraphQL just brings back the data that you've asked for, kind of like we mentioned just a moment ago. So, in a query, you might be able to say, OK, we have this GraphQL type of to-do, and this to-do has four fields. It has an ID, a name and a createdAtValue or fields. And this request, we want to get all four of those fields, so that's fine. But you can also say, I only want to get a subset or a smaller selection set of these fields. And this is going to continue to work. You don't have to kind of make any updates to your back end. The way that GraphQL is built, the way that these queries are created, you only get the data that you've asked for. So again, if you've ever worked with building out different types of client applications for a single back end in the modern world, you might have a web, a mobile, a desktop app that also maybe has an Apple Watch app or even a car app in the future, you never know. Having GraphQL as a data source allows you to kind of have a lot more control over your data access without having to change a lot of code on your back end, once the implementation is complete.
So next, let's talk about the GraphQL subscriptions, which are the real-time aspect of GraphQL. GraphQL subscriptions enable real-time communication between a client and an API. Subscriptions are event-based. So when you create, update, or delete an item, you might be able to subscribe to that event. So typically, a subscription might have a name like onCreate, onUpdate, or onDelete. So for a to-do app, you might have onCreate to do, onUpdate to do, etc. Subscriptions are typically a two-way connection. So you have to have a way to send the update and then have a way to receive the update back on the client. So a lot of times you are then asked about, okay, with this being said, how do you actually implement GraphQL subscriptions? So if you're using a service like Hasura or AppSync or any of these GraphQL services, this is going to be kind of taken care of for you. But if you're building your own API, you're typically going to be making the decision between something like servers and events or websockets.
6. GraphQL Subscriptions Overview
GraphQL APIs can be built using websockets. You can subscribe to subsets of data and receive only the fields you're interested in. For example, if you're interested in the ID, name, and completed value of a to-do, you can create a subscription to receive only that data.
I think a lot of the time you're seeing that GraphQL APIs are built using websockets. So, again, very similar to like a GraphQL query where you're asking for the data that you'd like and you only get that data, you can also subscribe to only subsets of data. So even though our to-do type has four different fields, we can make subscriptions and only return back like a subset of those fields. So let's say we only are interested in the ID, the name and the completed value, we'll create a subscription, we'll then only receive that data that we're subscribed to. So that's kind of a brief overview of GraphQL.
7. AppSync, CDK, and Infrastructure
AppSync is a managed GraphQL service that provides a consistent API layer for any AWS service or data source. It offers enterprise security features, including single or multi-authorization types. Adding GraphQL subscriptions is easy with AppSync. You can use AWS SDKs for AppSync or other libraries like Urql or Apollo. CDK is a new way to write infrastructure as code and is gaining popularity among Amplify users. It supports multiple cloud providers and can be used in combination with Amplify for building infrastructure.
Let's now talk about AppSync, and then we're going to talk about CDK and then we're going to do a live demo. So AppSync is a managed GraphQL service, and it's built using a lot of really scalable infrastructure on AWS. So AppSync provides a consistent API layer for any AWS service or any data source, and these data sources don't have to live on AWS, they can pretty much live anywhere as long as they have some type of HTTP access or some type of access to your data center, which is kind of also available as well.
We have enterprise security features built in. So if you need to have a single or multi-authorization types for a public and private access or a combination of the two, we have that built in. So IAM, we have OIDC for bringing your own authentication provider. So if you're using something like Auth0 or Octa, you use OIDC. We have Cognito for a managed authentication service, and then we have API keys for public access. And one of the really interesting things about AppSync is how easy it is to add a GraphQL subscription. And we're going to be walking through that in code in just a moment. But for any mutation, subscriptions are already built into the service. All you need to do is add one additional line of code, and you can subscribe to any type, but also any subset of that type. So for instance, you might think of a chat app where you have messages. You can also pass in arguments, up to five different arguments, to subscribe to only updates for something like a chat room, using the chat room ID.
And then we also build and maintain our own SDKs for all these AWS services, including AppSync. So you can use something like Urql or Apollo. But we have our own SDKs that manage a lot of things, like authorization headers. And we have SDKs and libraries for web, iOS, Android, React Native and Flutter.
8. Building an API with CDK and AppSync
To get started with CDK, install the CDK CLI and initialize a new project. Add additional features from the provided boilerplate. Create a new CDK project and install any necessary packages. Configure the API and code, including the GraphQL schema and base authorization. Add data sources and resolvers, mapping them to the schema. Define everything in code. In the live demo, we'll start from scratch in an empty CDK app, importing CDK, AppSync, DynamoDB, and Lambda.
You just have you install the CDK CLI, and then you initialize a new project and you're ready to go. And then you can start adding additional features from the boilerplate that's provided there. And we're going to kind of be starting off from a base project, and we're going to be looking at how that works.
So AppSync with CDK. What you'll typically do is you'll go ahead and create your new CDK project, just like we showed you just a moment ago, CDK Init. You'll install any NPM packages that you'll need, any CDK packages using NPM. So for instance, for AppSync, you're going to need to install the AppSync CDK package. And then you'll also maybe need to install other data sources like Lambda, DynamoDB for whatever databases you're interacting with from CDK. You'll then create and configure the API and code. So you'll give the API a name. You'll set your GraphQL schema, and then you'll set some base authorization configuration. And the API that we're going to build, we're going to be setting the base authorization configuration as public. You then add data sources, and then you add your resolvers to those data sources. So you might say, okay, I've created this API. I need to have a NoSQL database. I need a SQL database. And then I need to have a way to map my resolvers for the types and the fields and the operations of my schema to those data sources. And then you define all this stuff really in code. And we're going to be working with TypeScript today.
So I think the most interesting thing though, like we talk a lot about this stuff, is the actual live demo because we're going to be starting from scratch building this out. So what I'm going to do is I'm going to go into a project that I've created here. This is a really empty CDK app. I'm going to go ahead and open this up. And in the lib directory, we have the entry point for the CDK app. And this is basically going to be where we're writing our code. And if we go to package.JSON, we'll see that I've installed three dependencies. AppSync for CDK, DynamoDB, which is a NoSQL database and AWS Lambda. So using all these things, we're going to build out an API in just a couple of minutes. So what I'd like to do first is go ahead and get my imports. I'm going to import CDK, AppSync, DynamoDB and Lambda.
9. Creating API Reference and GraphQL Schema
We're going to create the API reference using the AppSync construct. The API name will be CDK Notes AppSync API. We'll set a default authorization configuration using API key and enable x-ray for additional logging. Then, we'll create the GraphQL schema for a notes app, including a note type, a query for listing notes, a mutation for creating notes, and a real-time subscription for on create note. The AWS_subscribe directive adds real-time functionality to the subscription.
And we're going to be using those to kind of create our API. Next, we're going to go ahead and create the API reference using the AppSync construct. And we're going to give the API a name. The API name is going to be CDK Notes AppSync API, and we'll call this like React Summit or something like that. I'll give a location for the GraphQL schema. This is going to be in GraphQL slash schema slash schema dot GraphQL, which we'll create in just a moment.
We then set a default authorization configuration. And for us, we're just going to be using API key. So we set that here and then we give an expiration and I'm setting that to expire 365 days from now. And then if you needed all multiple authorization types, we could have done that here. But we're not. And then I'm setting x-ray enable, because x-ray is basically a really sophisticated logging system that we can use for additional logging for debugging purposes.
The next thing we want to do is go ahead and create our GraphQL schema. So I'm going to go ahead and create a folder here and a schema dot GraphQL. And here we're going to go ahead and create our schema. And the schema that we're going to create is for a notes app. So we have a note type. We have a query for listing notes. We have a mutation for creating notes. And then we have the real time piece, which is the subscription here for on create note. So we're going to say we want to then have our, I'm sorry, at AWS underscore subscribe directive attached to this. And this is what adds the real time functionality. So like, if we look at the schema and we delete this, this is kind of all just basic GraphQL. This directive here is kind of app sync specific. But this is all you need to attach to any subscription to go ahead and set an array of mutations for the subscription to fire. No additional code is needed. So we're going to say we want to pass in an array, but the only item we're interested in is creating a note. We want to subscribe to that event. And then we're going to have this on create note subscription created for us. So I'm going to go ahead and close that.
10. Creating Lambda Function and Data Source
We're creating a Lambda function called notes Lambda with a runtime of node.js. The code will be located in the Lambda functions folder, with the entry point as main.handler. We're setting the memory size to improve performance. Then, we're adding the Lambda data source to the API using the notes Lambda reference.
And what we want to do now is go ahead and create a Lambda function. And the Lambda function is going to be where our business logic or our logic for interacting with the database lives. So we're going to go ahead and say we're creating a Lambda function. We're referencing that as notes Lambda. We're saying new Lambda function. And I might minimize this over here to kind of get a better look at the code. We're setting a runtime. Runtime is node.js. We're giving a location for our code, which is this Lambda functions folder, and then an entry point, which is main dot handler. And then we're setting memory size, bumping that up a little bit to make this more performant. And then what we're doing is we're taking that API reference. So we created our API up here. And we're saying we want to do an add Lambda data source, and then we're passing in this notes Lambda right here. So we create the API. We're setting a Lambda data source. Now we need to go ahead and create that Lambda function code.
11. Creating Lambda Resolvers
We're going to create a new folder called Lambda-fns to store our Lambda code. We'll add two resolvers, one for the ListNotes query and another for the create node mutation. These resolvers will pass the event into the Lambda function when called.
So what we're going to do is let's go ahead and create a new folder called Lambda-fns, and then that's where our Lambda code is going to live in just a moment. But for now, let's continue on. And what we want to do is we want to add our resolvers. And we need two resolvers because in our GraphQL schema, we have two operations. We have a query, and we have a mutation. The query is for ListNotes, so we're basically saying we want to resolve the ListNotes query into this function. So when this query is called, we're just passing that into the Lambda function in the event. And the same thing goes for the mutation. When a new node is created, when the create node resolver is fired, it's going to be passing the event into the Lambda function.
12. Creating DynamoDB Table and Lambda Functions
We create a DynamoDB table called NotesTable to store all our data. We configure it to be serverless and set a partition key. We grant access to the table from our Lambda function and set an environment variable to reference the table name. We've now created the API, function, and table, and minimized the code to about 40 lines. Next, we'll create our Lambda functions in a notes.ts file and define the note type. We'll use TypeScript and have two operations: creating a note and listing notes. The event object in the Lambda function will contain info and arguments, which we'll use in the function.
And then finally, what we need is some type of database to work with. So we're going to create a DynamoDB table. And the DynamoDB table is going to be where we store all of our data.
So I'm creating a new table called NotesTable using the DynamoDB CDK construct. We're setting some basic configuration, setting this to be serverless by using pay per request, and we're setting a partition key, which is basically the primary key of ID. And then we're going to go ahead and grant access to the DynamoDB table from our Lambda function, basically saying, OK, this function can interact with this table.
And then we're calling the add environment function, or we're setting an environment variable by calling NotesLambda.addEnvironment. And what we're doing is basically saying, OK, we want to be able to access an environment variable called NotesTable, but we don't know the actual value of that yet. So we're just going to kind of set the environment variable there because we don't know the value of the actual table name. But we know we need to access that table. This way, in the Lambda function, we can reference process.env.NotesTable.
So basically we created the API, we created a function, we mapped the operations from our API into that function, and then we created a database table and enabled access from our Lambda function to that table. We did all of that. And really, if we minimize this, it would be something like 40 lines of code or something or less.
So now that all that is created, we're really done with our code on our actual CDK project. Now all we need to do is create our Lambda functions. So I'm going to have a notes.ts file. And here, we're going to have our note type. This is just going to map pretty closely to our GraphQL type. We're just going to be using this for our TypeScripts here. And what we want to do now is have a main.js. Oops, this actually needs to be TypeScript, so I'm going to rename that to ts. And this function, we're going to have our two operations that we're going to be importing and using in just a moment. One for creating a note against DynamoDB, one for listing notes. We have a type that we're creating that has a couple of different fields. One is the info object that has the field name, and one is the arguments object that has the note. And we're going to be basically using these two pieces of data, I guess you could say, coming off of the event that's coming into the lambda function. So when this function is invoked, the event is going to have an event.info object. And it's going to have an event.arguments object. The arguments are the arguments passed into the function or into the operation.
13. Creating GraphQL Queries and Lambda Functions
So createNotes and GetNoteById are GraphQL queries that we switch on to execute different functions. We have createNote.ts for creating new items in DynamoDB and listNotes.ts for scanning the table and returning all data. The main function in the Lambda functions folder is referenced as handler. After building and deploying the CDK stack, we can test the API in the AWS console by creating a note with the mutation create note and returning the ID, name, and completed value.
So createNotes would have a notes argument. GetNoteById might have a note ID. And then the field name is the actual GraphQL query itself, so, or the GraphQL operation itself. So the field name is going to be referenced down here as createNote or listNotes. So we're going to be switching on that field name. So we're going to say, okay, if it's createNote, we want to execute this function. If it's listNotes, we want to execute that function.
So with that being said, we need to go ahead and create those last two functions. So we have createNote.ts, and we have listNotes.ts. This is a Lambda function for interacting with DynamoDB. So we're basically creating a params object, taking the note out of the argument, and we're calling the document, dynamodb.document.put, basically creating a new item. This is not really what we're focused on, though, but this is some pretty basic code for interacting with DynamoDB from a Lambda function. And then the other operation is listNotes, and this is a DynamoDB scan. So we're basically going to say the params we need is – the only thing we need to know is the table name, which is coming off of the process.environment variable, and we're calling the document.client.scan, passing in these params, and this is just going to pull everything out of that table and then return it back to us.
Okay, so after the stack has been deployed, we can now go ahead and test this out. So, I'm gonna go into the AWS console, and we're gonna go to the API we just created by going to AppSync. And we're gonna then look for the Notes app. So, we have the Notes app here. We can now go into our queries, and we can say, mutation create note. And then, we'll just go ahead and return the ID, name. And I think we also have the completed value, which we'll go ahead and set here as false.
14. Querying and Subscribing to Notes
Finally, we'll query the list of notes and set up a subscription for onCreateNote. This stack allows me to use my front-end skillset to build infrastructure code. CDK is a growing community that works well with Amplify. Check it out if you're interested in full stack cloud computing.
And then, finally, we'll just do a query of list notes. And here, we should go ahead and see the notes being returned from our API.
So, our API is up and running. Finally, we'll set up a subscription. And the subscription is going to basically be returning the same field, so let's copy those. And for onCreateNote, we want those. And now we have a subscription running, so if a new item is created, the subscription data would come through here.