Let's Talk GraphQL With Your Services

Hi, welcome everyone. So this is Roy at graphql Galaxy with Let's Talk graphql with your services. So what is this talk about? So during today, I will present you some use cases for graphql also with existing services. So let's have a look at legacy codes. It might look like this, right? Or is this actually pasta? I'm not sure if you know, but what is the last time you tried talking with pasta? For me, it was never. So let's assume we're going to be looking at the existing services as legacy code. So a little bit about myself. So my name is Roy. You can find me on Twitter with the Get Hack team. Currently I work for a renewable energy company called Vanderbron. Previously I worked for a lot of different companies, including the city of Amsterdam, most lately where we created open source projects for the city. And also you can find me online on YouTube either with conference videos from previous times or some of my books about react or react Native. And in the topic of today, my most current book about full stack graphql. So this is a book about how you can build graphql servers and clients with react, node.js and typescript. So if you have any questions afterwards, make sure to get it. So for today, we're going to be looking at how you can talk graphql with existing services. So suppose you start working on this great project, you just get hired at a new job, or maybe a new project starts at your current company. And this one is called the e-commerce platform to future. Probably you've been approached by recruiters that had something similar like this, like, hey, we're going to build the e-commerce platform to future. We want you to be part of it. Let's assume you fell into this trick and you started working at a company that is going to build this platform. So this might be the stack, right? So we have on the left, you can see an example UI and on the right, you can see technology. So it's using javascript and react, which all seems fine. And it's using a REST api with a database as a backend. So maybe after seeing this, you're thinking it's an e-commerce platform, but it might not be the e-commerce platform to future. If you think by now that's okay. If you think it's still the e-commerce platform of the future, that's also okay, because we might be able to find another way to use this platform and make it future proof. So, or you might also think the REST api and the database together are like the pretty bowl of spaghetti we saw before, which is the legacy code that you're going to be interacting with. So still we have some legacy code backend. We have javascript and react, and we somehow want to connect all this together with graphql, because if it's using REST, it's going to lead to several problems, right? Because this might be the REST api. You have three different requests or maybe even more. And all these requests will be called by your UI. So every single bits and every single bit and piece of our UI is using a separate REST api call, which in the end is going to be troublesome for people that are using mobile internet. And these often are directly correlated to database. So we have a REST api and we have a database, and those are closely linked together. So every table in our database will have a separate api call that will work like this in the UI. And all these api calls will also return some JSON. That might be a lot of JSON. And you're actually using small pieces of this JSON. So once you've seen this, you're probably thinking we should start using graphql for this, and that's exactly what we're going to be looking at today. So we don't want to do this, having all these separate requests that return a lot of JSON. It all needs to be going into our UI. That needs to be parsed. It needs to be normalized. That's something you absolutely want to prevent. So this can be done more efficiently, like we already saw. And this is, of course, with graphql. But the service isn't using graphql already. So let's try and find a way to have this backend service interact using graphql with our frontend UI. And let's do this in a way that we don't have to touch the legacy code or the pasta or spaghetti or whatsoever that's living there on the back. So what we want to prevent is having to touch anything that has been built by previous developers. So you have multiple options to do this. And I believe one of the best options is using existing libraries, because it's going to get rid of a lot of stuff. It's going to prevent you from making mistakes. It's going to prevent you from having to maintain another service. And for this, I would definitely advise you to use graphql Mesh. I believe Uri Goldstein is also a speaker today. So he'll be, or maybe already presented, I'm not sure. At least he'll be probably, he'll be talking a bit about graphql Mesh and how you can use it. And if you want to stay away from making any custom code, it's definitely a way to go. But for today, we're going to be looking at a different approach. And this is by creating your own data layer or data access layer. And using this layer, you'll be able to use graphql for your backend services without having to change those services. And it's similar to what graphql Mesh does, except we're going to be owning the entire data flow. We're going to be owning all the logic in there. And in case you're not familiar, a data access layer or data layer is a layer of a computer program that provides simplified access to the data of that service. So what we'll be doing, we'll be building a data layer that provides simplified access using graphql over our legacy code. So this will help us do a lot of different things like caching, like monitoring, but also it's going to prevent us from making mistakes by making changes to legacy code that runs, but somehow nobody knows how it runs, which often is the case with legacy code, of course. So what, this is a question you might ask yourself, why would I be creating a layer on top of a service that already exists, that runs perfectly fine? Why should we do that? Right? Because is my api already a data layer because probably your legacy code already provides an api with an interface that's going to give you all the data that you need. But we're going to be creating a data layer on top of a data layer. So this, you might feel like inception right now because yeah, somehow we're creating something on top of something, or maybe it's already inside something. So something to think about for the rest of this talk. So how do we get from a graphql document to data? Because that's what we want, right? We want to send the graphql document to our server and we want the server to return the data from our legacy code. And it might look like this, right? So we saw the UI previously, that's the three separate REST api calls. And what we want to do actually, instead of sending those REST api calls, we want to get, we want to send a query and we want to return this data in a way that we formatted it, because that's what graphql does. So somehow we want to be going from a document, which you see on the right, to the data that we have on the left. And this is perfectly fine because this is what graphql does, and this is how graphql is going to do it for us. So let's see how we can actually implement that. That's by doing this. So the first step is send a graphql document, which you can see already graded out. And this graphql document will be converted to an abstract syntax tree. So basically what is this? This is a representation of whatever document you're sending. So we're sending a document that contains an operation, which is a query. It contains fields like top level fields, which is products, but also lower level fields like title or relationships with categories. All this will be parsed into an abstract syntax tree that can be used by the graphql schema and the graphql server to find out how these things match. So your server will be in your schema level of Vita-Wallet here, because there will be information in there about resolvers, about connections, all these things. This will be described in your schema and also in the server in general. And then the final thing that will happen is the resolvers will retrieve the data. So based on the document that you send, it will be converted into an ASD and that ASD will be matched to your schema. And then finally, it will end up at your resolvers that will retrieve the data. So the document that you initially sent to the server will be responded with a JSON output with exactly that data. So this is briefly how it should work and how it will work with graphql. And if you use existing libraries like the graphql Mesh thing, it will already do this for you without any trouble. But for today, we're going to be looking at how we can build this ourself with the custom code, which in the end, I think is the way to go because obstruction layers are great, but they also give you the risk of ending up with even more legacy codes. So it's something you should always think about. Am I going to do this myself or will I be using an existing library that already does one of the heavy lifting for me? And constructing the data layer around the data works the other way around, right? Because now, my starting point was the document, went to the AppSecStringTextTree, went to the schema and the server, and in the end, resolvers got the data. But if you want to have a data layer, actually your data is the starting point because we need to have the data in order to construct resolvers and the schema so you actually have a server to send the document to. Because we will need to serve our otherwise we can't possibly use graphql operations like the query we saw before to get our data. And the constructing the data layer, like I told you, it's exactly the other way around. So we have our data as a starting point, and we will end up with a server that can accept documents and that will receive our data in the end. And for this, there are multiple ways you can go. So you might have heard about code first or resolver first. Usually they're the same two things. Basically, it means that you don't really construct a schema as your starting point, but you'll be thinking about existing code, existing resolvers, how to handle or retrieve your data. Another way we could do is schema first. So both could be options like the code or resolver first option or schema first. And it all depends what you want to see as a starting point. So more about it later, but I think the code and resolver first discussion is a good thing to have or schema first, whenever you would pick up such a project with your team. Because you need to be able to know how to handle the data, how to handle data flows, and whether you'll be working with a code or resolver first solution, or maybe a schema first solution. And the starting point should always be the source of truth for your data. So if your existing data is a REST api or a legacy api, then this will be the source of truth. So your api might have a Swagger definitions that you can use or any other open api definitions. It might return some sort of schema in the form of JSON, maybe with the JSON schema. If it's a database, you might have migration documents, you may be with mongodb, you might have Mongoose models, same for any other kind of database. So the starting point when doing this should be any source of truth for your data. And if you don't have one, it's going to be a bit tougher. And if you don't have a source of truth for your data, I would always advise to go schema first because then your schema will be the source of truth for your data. And if you already have a source of truth, you can use a Swagger, open api or Mongoose models to create the resolvers code first. So the thing we need to do is map the data specification to a graphql schema. And this is easier than you think, because there are packages to do this for like the graphql mesh we saw before. But if there isn't, and actually I found a use case myself because quite recently, we started working at a Salesforce implementation. And if you know Salesforce, they have REST APIs, but they don't come with Swagger. So they come with their own describe endpoint, which is sort of following the JSON api schema. So we needed to create everything ourselves, but we had a source of truth. So that's the important part. Without a source of truth, it will be very difficult to do this. And you should always go for a schema first approach. But if you don't, you can just go for a code or resolve a first approach. But the thing you need to do is use the source of truth, whether or not you have it to specify a graphql schema, whether or not through code first, which we define the resolvers that create our schema or by directly creating the schema. And this might look like this, right? Because previously we saw we have a REST api, which is our legacy project. And we want to be able to have a data layer on top of this REST api to get our data in graphql. And at the forefront of this database, of this REST api, there's also a database. So there are two ways you can go here. So either you're going to be building on top of your REST api, or you're going to be building on top of your database. And which of the you should select really depends on a lot of things. But you can imagine that if your REST api is a one-to-one mapping of the database without any normalizations, you should use the database as your source of truth. But if your REST api is having side effects, so instead of just getting data from the database, it also do normalizations, maybe call other APIs, then we should definitely take the REST api as a sort of truth. And if you're lucky, this REST api will have a Swagger definition or a JSON schema. And if it doesn't, you're going to be actually checking all the endpoints that you want to use and create a schema out of these endpoints. So it might look something like this, because we have a REST api with some sort of definition. And you might want to port this to a schema or to some sort of schema definition. So for this, I actually went with the graphql.js approach by using graphql object types to define my schema like this. And in here, I can also define resolvers or variables that I want to accept. But it's just like the basic mapping. And this mapping is done with the knowledge that we might not have a source of truth for this api, because the source of truth might be outdated, something you often see with projects that use Swagger. It's a lot of manual work if you don't use the proper libraries. So let's assume the only thing we have is the request and maybe some minor documentation on our side. So you would have to find a way to map this to graphql object types, or to a schema, or to any sort of library that uses a code-first approach. But for this time, we just created manually like this. So an example of our project, we use the Salesforce described endpoints with a JSON schema to create the graphql schema out of it. So basically it looks like this. So the mapping is one-to-one, and you just insert the fields that you might want to use. And once you've done this, you'll be able to create resolvers to retrieve that data from our original source. So in this solution, it will be a REST api, or in my solution, it could be Salesforce. But even if you're using a database, but maybe mongodb and Mongoose, then the resolvers will be retrieving data from the database. So your resolvers should be able to retrieve the data that matches the schema. And basically it looks like this. So I went for an old, nice approach with graphql object types to create my schema. And the schema can also include resolvers. So in here, you can see I've created one for product that relates to my product REST api endpoints. And there it is, resolver that will call some sort of class or method or function to get the product based on an ID that I get from the resolver. So resolver is getting some information. It's getting the parent object, the arguments, the context, and also the info object. And the info object can also be quite nice to explore more in detail if you're going for this approach, because the info object will always know all the information you have about the AST, so the abstract syntax tree. So it's interesting to have a look at this later. But yeah, so fortunately we don't have the time to do this today, but I believe there's many more things to investigate for that. And I created another one for category. And you can see it is using the parent object to get the category for a certain product. So this is about the resolver. So once you've got the schema and you have a source of truth that you created using the schema, you can also use this schema to create resolvers at once. And if you're going for different solutions, so maybe you're using typescript, you might want to go for the code or resolver first approach. And then the schema will automatically be generated once you create resolvers. So there are multiple ways to approach this. So I think an important question to ask yourself is which solution should I choose? So I think I give you a very broad oversight about how you can use existing services and make them talk graphql. Of course, I mentioned a library called the graphql Mesh, which Yuri probably is mentioning today as well, or maybe already has mentioned it. And this is great if you have an existing source of truth and you're pretty sure you don't want to make that much customizations to it. Although there are options to add custom resolvers. But in general, I think my approach was you should go with your existing legacy code, take a source of truth, and then try to find out what you want to use or not want to reuse. And then how you can create a schema out of it, because in the end, your schema or your resolvers should be defining the data model for a graphql server. And not all servers really like this approach because you may be using older things like SOAP or if you are using REST and you don't have a Swagger definition, then you should do this all by yourself, which I think can also be done manually, can be automated if you use the response from every endpoint. And this is a very good discussion to have with your team whenever you start working on a project like this. So you want to build a data layer on top of your legacy code, but you don't want to create more legacy code. So if you do it all custom, then you might create a lot of legacy code. But if you invest in an existing library, then you have less customizations that you can do. And also, if you want to have side effects in your library, then of course, you need to build a custom one yourself. But these are very good things to discuss beforehand, but hopefully you will get some sort of idea how you can use a source of truth to create a graphql layer on top of it. And for me personally, the approach with Salesforce required a lot of customizations, but in the end, I think it's also a nice approach to use and to be able to use it for all those services. And in the end, as soon as you build a graph, sort of one sort of legacy code, you of course can use schema stitching or federations to connect these services with other services as well. So if you want to learn more about this, I believe we have some time for Q&A. Or you can just go to my Twitter page and ask me a question over there. And also make sure to find me on YouTube. And of course, if you want to know more about graphql, make sure to find my book online because there's a lot of cool content on there. You can get the first chapter for free and see whatever or not it will suit you. Or again, just find me online and ask the questions over there. So yes, thanks a lot for your time. Hi. Hi, Rai. Hey. So my first idea was actually also notepad+++. Oh, you only got three pluses in that one. Yeah, I was just thinking. I think the third plus was when I found out you could also link FTP to it. You could directly push your code to the server. That sounds pretty good. Yeah, when you could write to production, that was great. In many ways, a simpler time. Yeah, those were the times. Just refresh production and see your changes. And how did you get inside the server room? So yeah, what inspired you to make this talk about incorporating graphql onto a legacy stack? Working with legacy code. I think everyone did. And yeah, for me myself, the last project I did was mostly frontends. And we wanted to change things on the back end, but the back end still didn't want to change anything. So we ended up building something around it. Yeah, I think that's a really common use case. It wasn't until quite recently, actually, I made a graphql system from scratch, which wrote directly to a database. It's always been a case of, look, we've got seven different disparate systems. And wouldn't it be great if there was a graph to connect them? Also, they all have different authentication. Yeah, that sounds interesting. The classical problems. Yeah. So what do you think are the most common struggles people have when they're trying to bring a data layer on top of existing services? In your experience, what has been the top thing that causes grief? Like when implementing it or reasons to implement it? Oh, when implementing it. I think one of the biggest things is authentication and caching. authentication probably because the server that you're wrapping, of course, you'd understand it. Caching because you want to make sure that it doesn't get out of sync, so you want to do some sort of caching, but you also don't want your data to be outdated. So if you're working with legacy and they don't provide e-text or something, it's going to be pretty hard to make sure that you have recent data. Excellent. Do you think there's any sort of new tech or tooling coming out that you think will make this process easier? Yeah, there's a lot of things. So I believe Yuri Goldstein also gave a talk and he created something called graphql Mesh, which is great for this. And of course, you could also use azure and I believe some other services doing this as well and they all automate it for you, but then again, it's something you might have to pay for and it's also fun to build it yourself. Cool. Well, let's take a question from Juan. In the example you showed us, we would still receive the whole data set from the RESTful core. The difference would be that the whole data will be received in a middle place, which in return only was asked from the UI through graphql, correct? Yeah, that's correct. So performance-wise, it might not be the best improvement, but I think what I wanted to show is most of all, you can use the developer experience and the tooling around graphql instead of making the most performance server because to make it most performance, you of course have to change the legacy code, which is something you usually can't. And there was a follow-up question to this question, which is, in this case, when would you add this middle section and when would you not? When would you add it? I think it depends on the kind of normalizations you need to do. So previously, we did an open source project for the city of Amsterdam and all the APIs that we consumed were also open source. So they were made available for a broad range of things. So some people were using it for machine to machine, other people were using some parts of it. And the platform rebuild took 20 of these APIs with 80 different calls and normalized, merged them all together. So these are things we didn't want to do on the backend, but also things you don't want to do on the frontend because, I don't know, if you have to parse 20 different api calls, merge the data together, it's really something you want to prevent it from doing. Sorry, I had a phone call to mute. I'm so sorry. Okay, well, thank you for that question. I'm sorry for my rudeness. I should have put that in silent a long time ago. It's actually my iPad that rang because I put my phone on silent, but apparently if you have an iPhone, it also rings your iPad. Yeah, that's really annoying. And it takes long to go. So when my phone rings and at the end of my AirPods, it's still MacBooks and iPods, iPads, and then the phone going off. It's really annoying. I'm going to quickly turn this off. So yeah, when you sell this idea, or I guess I should rephrase this. Do you think, do you have any tips for selling this approach, this technology of a competing ways of managing your services with you're trying to get a customer interested in this technology stack? Yeah, that's a question with many answers, I think. But usually I think like using graphql for client facing applications, I think it's something you have to do. Because if you're trying to force a REST api to a client facing application, I think it's something, well, I don't think it has any benefits for everyone. Okay, wonderful. Thank you for these answers. I have exhausted my pile of questions. And we have no more questions from the audience. So this is your slot at the end. Is there anything you would like to promote, plug, tell the audience? Not really. I think if you saw my talk, then hopefully you'll be starting to use graphql. And hopefully you'll be starting to using it from scratch, and just connect your database to it. So you get the optimal experience. I hope so too. We should all try graphql today. All right, thank you very much. Leave your claps in the chat for Roy. And thank you again for coming on. Yes, bye bye. Bye bye.