Database Access on the Edge with Cloudflare Workers & Prisma

Bookmark

Edge functions are pushing the limit of serverless computing – but with new tools, come new challenges. Due to their limitations, edge functions don't allow talking to popular databases like PostgreSQL and MySQL. In this talk, you will learn how you can connect and interact with your database from Cloudflare Workers using the Prisma Data Proxy.


You can check the slides for Alex's talkĀ here.Ā 



Transcription


How to use cloudflare Workers with Alex How to use cloudflare Workers with Alex How to use cloudflare Workers with Alex Hello, friends. Today I'll be talking about database access on the edge with cloudflare Workers. My name is Alex and I'm a developer advocate at prisma. Feel free to reach out to me after the talk if you have any questions or just any random questions or connect. So this talk will be divided into four sections. The first part will be about cloudflare Workers or the edge. We'll discuss what it is, how different it is from the traditional serverless platforms and the growing pains of serverless. The second section will take a look at prisma and the prisma data proxy exploring how the data proxy helps you interact with your database on the edge. And the third part or the third section will be the exciting bit where we will have a demo. We'll migrate an application using a JSON file for storing and retrieving quotes and move over the data to a database and deploy it to the edge. And finally, we'll have a recap of the talk on the end. And for now, let's jump into it. So what is the edge? It seems to be a buzzword that's used a lot, but what does it mean? The edge is a form of serverless compute that's distributed and in the context of cloudflare Workers, it's deployed to cloudflare's network of data centers. Now that you understand what the edge is and how cloudflare, let's take a look at how cloudflare Workers are different from traditional serverless. As I mentioned before, it's distributed. So traditional serverless functions are deployed to specific data centers or region. And in contrast, cloudflare Workers run your application code much closer to your user, which could even be in the same city. cloudflare Workers are also significantly lighter because the platform is built on top of the V8 engine. It's the same engine that powers Chromium-based browsers such as Chrome and Edge and also the node.js runtime. So whenever your cloudflare Worker is invoked, instead of provisioning an entire container for your serverless function, cloudflare Worker spins up a V8 isolate that is significantly smaller. And if you've heard about the cold start problem on serverless, then you understand why this is an exciting technology. So cloudflare Workers also provide higher performance. This is adding on top of the faster points. It's because the environment is significantly lighter, which leads to a shorter startup time or even possibly none. And you also would experience lower latency whenever you invoke your function because it would be invoked from the data center that is closest to you. Now, I know this all sounds delightful, but cloudflare Workers still suffer from the same problems that traditional serverless functions do in the context of databases. So connecting to databases is still a challenge from cloudflare Workers. This is because V8 engines support sending only HTTP-based connections. However, traditional databases depend on long-lived TCP connections. This becomes a bummer. The different databases present different implementations of the interfaces of how you could connect to them, which is all chaotic and needs on. database connections are also stateful, while serverless environments are stateless, meaning that you can easily run out of database connections because every worker creates a new connection pool. And when you run out of connections, requests in your application would start failing. So how can you solve this problem? Well, you have a number of options available to you, just like in the rest of the javascript ecosystem. One option would be to go with cloudflare-specific solutions, such as workers key value store and durable objects. The second option, if you have an existing database, you could use Postgres, which converts your existing Postgres database into a RESTful api. Another option would be to use the Postgres and MySQL drivers and modifying them to use HTTP using deno, I believe. And finally, the gist of this talk, which is the prisma, the prisma data proxy. And this talk will focus on this, but feel free to try out the other options for yourself as well. So prisma is a next-gen type-safe ORM for node.js that works with both javascript and typescript. It supports a number of databases, such as SQLite, MySQL, Postgres, SQL Server, and also CockroachDB and mongodb, which are still in preview. The core benefits of prisma is that it boosts productivity by letting developers query data in natural and familiar ways and also providing a human-readable data model. prisma also increases developer confidence with the type safety, the auto-completion, and the robust api for interacting with your database. So even if you don't use typescript in your project, most editors will provide auto-completion for your database queries when you're using prisma. And the prisma toolkit comprises of three tools. prisma Client, that's the type-safe and auto-generated client. prisma Migrate, that's a declarative data modeling and data migration tool. It generates fully customizable SQL database schema migrations based off of your prisma schema. And the third component is prisma Studio, that's a modern and intuitive graphical user interface for your database. And now this brings us to the prisma data Proxy. It's an intermediary between your application and your serverless, your application or serverless function and your database. It helps you manage connection pools by ensuring that you don't run, exhaust your connection limits that would lead to the failed requests. So when you query your database, the prisma Client would connect to your database via the data proxy that would use HTTP. Now this is exciting because it enables you to bring all the good parts of your trusted relational database and the prisma Developer tooling to the edge. You can still adopt cloudflare specific data stores like workers KV and durable objects for specific cases where it makes sense, while benefiting from querying your central database for most of your app. Now this brings us to the most exciting bit of the talk, which is the demo. What we'll be doing is converting the application that is using a JSON file to retrieve and create quotes or get getting quotes by IDs and we'll move that application. We'll migrate the data to a database and convert the existing routes to start using prisma to query the data. Now let's jump into the demo. I've already set up the base template for the baseline for the project. It has three specific routes in the index.ts, which is the router.getQuotes over here. We can get quotes by ID and we also make a post, which doesn't really create a new quote, but it just sends back what you sent. The project is also using webpack. It has a minimal webpack configuration for this to get running. I'll go ahead and comment on the prisma alias, which we'll use in a later section for the demo. The prisma alias will allow you to, will help you resolve the prisma client dependency correctly when you're querying a database. So let's get into it. So the first step to set up our project is to set up our project by adding prisma. So I'll go ahead and install the prisma CLI as a dev dependency and then install the prisma client as a normal dependency. Wait that to load for a moment. Now that we've installed the prisma client dependency, I'll go ahead and initialize our project to start using prisma by running the npx prisma init command. So what this command does is create a new directory in our project called prisma with a schema.prisma file. This schema.prisma file will be used to define our data models, which would be applied to our database. The command also generates, creates a.env file where we'll store our environment variables for the project. So for now, I'll go ahead and add in, paste in my connection string to my database. Now that's done, I'll go over to the schema.prisma file and start using the data proxy in your project. You can enable it by adding the preview feature flag in the generator block in your schema.prisma file. data proxy. That's done. I'll close this, expand this. Now let's go ahead and model our data. Our code, as we can see over in the data.json contains three fields, three properties, which is the ID, the content, and the author. I got this from an api on the internet, but this will be in the starter repository that I'll share before the end of this talk. So we'll create our new model code, which will have an ID, that's a string, and it's an ID value. And by default, we want this to be generated. We also have the content property, the content field, which will be a type string. We also want to have the author, which is a string. And if you'd like, you can add a created field in your table, which will be a timestamp of when the record was added to the database. And we want this to be auto-generated using the now function. So now that we've modeled, let's apply this schema, or the schema that we just created to our database. I'll run a npx prisma db push command to apply whatever we have to our database. Yes, it's got existing data. I'll go ahead and drop it. The next step is to migrate or seed the database with the codes that we have in the data.json file. So inside the schema.prisma, inside the prisma folder, I'll add a seed.ts file. And in here, I will paste in some code, which I'll explain in a moment. So what this does is import the prisma client from our node modules, and we are also importing the quotes from the data.json file. And since it's an array of roughly 500 quotes, we'll loop through each and every quote and insert a record into our database inside this block. The next step now is to execute this file again so that we can actually see the data. So I'll use ts node to execute our typescript file, so npx.ts node prisma seed.ts. Give it a moment. And it's done. So, so far, we've initialized our project with prisma, and we migrated our existing data from the JSON file onto the database. So the next step is to set up our project on the prisma data platform so that we can get an api key or a connection string that will allow us to communicate with our database using HTTP. So the first step is I'll go over to GitHub and create a new repository. I'll call it prisma CFW, short form for cloud Flow Workers. I'll make it private, create it. So locally, I'll commit my files to GitHub first, but I'll first start with initializing it, getting it, get add, and then we'll commit all our files. Now that's done, we'll go ahead and import our project on the prisma data platform, but first let's check whether the changes were synced. Oh, I have, I missed a step where we needed to add an origin first to, so before we can actually push it to GitHub. So I'll paste in the command from GitHub. The changes have been pushed over to GitHub, so if I refresh this, we should be able to see our new changes, the changes we've just made. Cool. That's good progress. So the next step is to import the project on the prisma data platform. So I'll go ahead and click new project. I'll give it a name, prisma CFW. We'll import the repository, prisma CFW. That's all good. I'll click next, continue. I'll be using my own database, and I'll paste in my connection string as well. So the prisma data proxy will be generated once we complete setting up the project on the data platform. Now that's done. So you're given two connection strings once the project is done setting up. So the first one will be the data proxy api key, which I will copy. And the second one is your connection string to your database. I am done, so I'll click done here, and we are all good. So let's go back to our project and in the.env file, I will create a duplicate of the database URL environment variable and paste the one we just created, but comment out the first one. Give it a yes. Cool. Now that's done, it's time to refactor our application. Sorry, not now. So back in our index.ts file, we will replace our data.json import with the importing prisma client. And then we'll create a new instance of the prisma client. And that's fast refactor, fast route. So we'll make a query to get all the quotes from our database. So const quotes equals to await prisma.quote.findMany. That's done. Let's refactor the getting a quote by the ID. So const result equals to await prisma.quote.findUnique. And then we need to provide a where clause that requires the ID, which is from the params value over here. Lastly, we'll need to be able to create a new record. So over here, we'll await prisma.quote.create. And we will, one moment, we need to pass in the data property, which requires the content and the author values, which can be from the request body. Now that's done, the next step is to regenerate our prisma client so that it will be able to use the prisma data proxy. Now that you're done refactoring the application, let's now go ahead and test out the application by running the wrangler dev command. That's ready. I'll switch over to Postman and increase my font. And this I already pasted in the command to fetch all the quotes. This works the first time you make a request. It's sort of slow, but over time, the request time reduces. So let's copy an ID and over here in the getByID, let's replace this with our new ID and request for it. And yes, yes, it works. And let's create a new quote, which I said, that the edge rhymes with the ledge, because I think that's a cool quote. And boom, that works. We got our response. And congratulations, we've finally migrated the application. Now, the last step is to deploy our application to cloudflare workers by running the wrangler publish command. As I mentioned before, you can notice here that the project is only 97 kilobytes big and relatively takes a short time. So go ahead and copy this command, this URL. Go back to cloudflare workers. Go back to Postman and then replace this URL with the one that's in production. And boom, application works. We've successfully migrated it. Now let's go back to the slides. And if you'd like to build the same application or refactor the application, it's available on GitHub. You can tune it. And if you'd also like to play around with the application, the live demo is available on cloudflare workers deployed. And as you can see, the whole experience building and quickly migrating felt relatively fast. In about 10 minutes, we were able to successfully complete the whole migration process. Now, with a few final thoughts, the edge is still in its relatively early stages. And this workflow provides one of the easiest or the simplest setups to get up and running. It would be useful if your data proxy, given that it's deployed to a specific region, is closer to your database and also your application deployed closer to your users. The edge is a new paradigm that makes you reason differently about data because data is distributed. And when it comes to relational databases, distributing this data is sort of a computer science problem because maintaining consistency across the globe, if you have it deployed to multiple regions, is a challenge. So the advantage of using prisma is that the data proxy is deployed closer to your centralized database. So you can use your normal development workflows to build your application. And this talk is a final recap, cover the challenges and good parts of the challenges that exist when working with cloudflare workers or serverless platforms. We also covered the advantages that cloudflare workers have over other platforms. We also discussed what prisma is and what the prisma data proxy is and the problem that it solves. And finally, we added database access to an application and then deployed it to cloudflare workers. Thank you for listening. I hope you found the talk useful and feel free to reach out to me if you have any questions. This is an exciting time to be working on the edge. So Alex asks, what database are you using with your serverless apps? And we see 44% answered PostgresQL. I don't even know if that's how you pronounce it. I never said it out loud. Do I say it correctly, Alex? I'm not sure it's PostgresQL, Postgres, depends on how you prefer to say it. But mostly, probably Postgres. Oh, just Postgres. OK. What do you think about this result? Is it what you were expecting? Well, it's changing. I'm not completely sure. Given that also PlanetScale is new, a lot of folks might be moving over to it. So it's more of a MySQL database. But yeah, it's close to what I was expecting. Yeah. Yeah. Nice. OK. So let's jump into the audience questions. Let me open to the Discord server. CCC Rish is asking, from what I understood, the prisma proxy is near the edge, right? But what about the real database? It's not exactly near the edge. Right now, the data proxy is deployed to a specific region, and you'd rather deploy that as close as you can to your database. So it's not completely edge ready, but the perfect use case would be if your users are in, for example, in a specific city. But hopefully, as the edge matures, we can improve the data proxy as well as just the database access on the edge. So a little follow up question for me. So I noticed that you said when the edge improved and you said cities, do you know if there's actually like countries that have multiple edge locations? And even like I can imagine a country like Germany, you're in Germany, that you would have a server from the same company in, say, Hamburg and one in Berlin and one in Frankfurt. Do you know if that's also a thing? Or is it like there's a German server? I'd say yes. I think companies like cloudflare have their data centers all over the globe, and I'm sure that probably multiple data centers in Germany, for example. So, yes, it's possible. Yeah, that's super cool. Yeah. Things are speeding up. Nice. Thanks. Next question is from Hail to the Wood. How would you suggest about going about migrating a larger scale application towards using something like prisma proxy, for example, one that is currently using serverless functions, talking directly to DynamoDB? prisma doesn't currently support DynamoDB. So this would be a bigger challenge. I'm not sure we have any best practices, but right now the prisma data proxy is also in early access. So we wouldn't recommend you to use it in production. However, you can still use it to just experiment, provide feedback so that we can also prepare it to be generally available to everyone. Yeah. OK. Thanks. Then another question. What's the difference between serverless, CDN and the edge? So this is a similar but still different concept. The serverless is like a deployment architecture where you don't really have to worry about your infrastructure at all or just managing any servers that are involved when you deploy applications. Now CDNs or content delivery networks are a network of data centers that are deployed all over the globe. For example, Netlify and cloudflare, where you can run your application as close as possible to your user. And the edge takes this a step further and deploys you. Now you can run sort of computation logic now as close as possible to your users. I hope that's a good enough explanation for our viewers. Yeah. So to debrief that, a CDN is the same as an edge server, but just less physical locations. It's almost the same in that CDNs are mostly used to serve assets or deploy applications. We're already using it with the application when you deploy to Netlify and they have their data centers all over. So it's mostly been used to serve, for example, static assets. I could be wrong and feel free to correct me on that. Yes. No, that's also what I understand, but there's always smarter people. So if you want to correct us, do so on the Discord server. Yeah. For now, we don't have any questions. So if you have any questions, be sure to type them. And to give you some time, I will ask Alex about the Witcher. So Alex, we were just discussing this. Can you give me your best representation of Henry Cavill's brrr? My throat is a little sore right now. So I can just say brrr. I think it's pretty good. I hope so. I hope so. You sound properly Witcher level annoyed. Of course. Yes. Yeah. All right. So there's no questions coming in. I see no one is typing at the moment. So we are just going to cut a little bit shorter to the break. So Alex, I will release you. If you want to talk to Alex, you can do so on the spatial chat. Alex is going to jump there now. So if you want to continue the discussion about prisma or database access on the edge with cloudflare Workers in prisma, you can do so on spatial chat. So Alex, thanks a lot for being with us here today and enjoy the rest of your day. Thank you for having me too.
31 min
17 Feb, 2022

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Workshops on related topic