Panel Discussion: GraphQL Working Group

Thanks for having us here. And I'm really excited for this panel discussion. I think the subject is really interesting and something that should be highlighted more. And it's really important. So I'm really excited for the opportunity. I'll introduce some of the amazing panelists that I have here today that I hope to talk as much less as I can and give them the stage. First of all, Brielle Harrison is a senior principal manager at PayPal and has been working with graphql since she worked as an engineer at Facebook back in 2013 till 2015. She also serves on the graphql Technical Steering Committee. We have Benji, he's a community-founded open source developer, maintainer of PostGraphQL and a member of the graphql Technical Steering Committee. Lee Byron is one of the co-creators of graphql and the director of the graphql Foundation, as well as an engineering manager at Robinhood. And Sasha Salmon, she's a staff software engineer and co-tech lead at the platform team at Twitter. She also serves at the Technical Steering Committee and the graphql Foundation. So a lot of amazing people here. And we have a lot to talk about. So maybe I'll start with a basic question, but I think a very interesting one, which is what is actually the graphql working group and why do we actually need it? Anyone wants to take that first? I can kick this one off. So working group, what is it? Why do we need it? Where did it come from? The working group and the Technical Steering Committee are actually two related but different things. And the working group is a room, virtual room, just kind of like this one, where a bunch of people who care about how graphql develops over time get to meet once a month and talk about what they're working on. And we try to make progress and we work through problems together. And ultimately a lot of those problems get merged into the graphql spec and get reflected back out in a lot of the software that all of you get to use. But that's not the model that all open source projects use. It's certainly not the model that all projects use. And so it was not obvious that we would need this thing. So I think it's maybe a little helpful to just hear a little bit of the history of the project, maybe through the lens of how it gets worked on rather than what it is. So graphql was started in 2012 and it was just kind of a crazy idea at that point. So the working form there was like a couple of people writing pull requests, or at Facebook we called them diffs, and code review. So most of the actual work happened via code review. And that was the working model for about three years, as it probably is for many of you, and all of the work that you do day to day in your jobs. We were getting ready to open source graphql in 2015. And in the process of that, made some fairly significant changes. And the way we talked about those changes was as proposals. So I would write a proposal to change the syntax in some significant way. I would write a document and then I would hand that to my coworkers and they would give me feedback. You might do this as design docs with your team. If you're going to make a big change to a project that you work on, you want to write down what you want to do before you do it so you can get good feedback. And that was kind of my mental model at the time. And I wasn't the only one writing proposals. As we did that, and then as we open sourced, we had this sort of phase shift where now people were writing pull requests to us. We had all the code up on GitHub. So we would get these pull requests, added some crazy feature to graphql, please review my diff. We're like, whoa, we had already sort of progressed from like, please review my diff in 2012, three years ago, to like design docs with formal review internally. Like this was an important internal project for us that we were now making publicly available. So we had to sort of like shift how people were interacting with our open source project to say like, actually, can you please write down what you want to do? Like do a design doc, we'll do a review. And the working group was emergent from this. We realized that some of those things were fairly straightforward. We could do them all asynchronously, we could comment on your diff. But your diff kind of started with like a big block at the top that described what you're going to do. Other folks needed a lot more feedback. And so we would set up live calls, actually. And for a few of those, we would drive to wherever their office was, and we would talk to them in person. And we'd figure out what it is they wanted to do. And we would, you know, old school style, go in a room with a whiteboard and a marker and like draw stuff until we all got confident about what problem we were solving and why. And then this started to scale. There were a lot more people who are contributing, there were a lot more people who are using graphql. And they wanted to have a say. They wanted to make sure that if a significant change was going to happen, that they could put in their two cents and make sure that it would work. And of course, the core of us working on graphql, we wanted to make sure that any changes we made, we had confidence, not just work for us at our company, but would work for everybody else at their companies too. And so it started to get more and more formalized. And it started with a bit of an ad hoc frequency. We would do them when we needed them. And I think at some point we had roughly six or seven of them a year. We decided to lock them to the calendar and do them every month. And I think we decided to start doing the monthly three and a half or four years ago. So at this point now, it's a healthy, long running meeting with a handful of very familiar faces, but with a constant rotation of cast of characters of people who have something interesting that they're working on. They bring it in to play with us. Thanks. This is really interesting. And I'm very lucky to also join some of those meetings. I would love to hear from others. What was your experience when you joined and what did you get out of it? And Brielle, maybe you could share yours. Sure. Just checking to see if I'm on mute. Yeah. So it's interesting. I have a split opinion on this, having worked at Facebook before it was open sourced and then actually trying to implement it in a company after it was. And so one of the big things here I think that's really important is that when you're working at some of these companies that go to adopt it, people are always trying to integrate graphql into their company's infrastructure. And they have a lot of different ideas and opinions about this, especially if they were very heavily rest focused or any of the other kinds of things that might have gone on there. And so when we first brought it to PayPal, there was a lot of discussion about like, you know, we always handle errors in a certain way, but graphql does it different. And how should we make that go? And you know, there was lots of very heated discussions about how we should implement things. One of the things that we had to do there is we almost had to put a committee together inside of the company in order to just basically have some sort of sanity because there were too many people on all sides making this go and everybody wanted to solve it for their local space. The working group is really good because when you have this, you can come in and you can propose what works really well for you, for your own company, and try to bring those ideas out. Inevitably, you almost always find that what works well for you doesn't always work well for everybody else. And that's why that type of a group is really important to have. I think there's a lot of anecdotal stories that we don't have time to talk about here that are about the things that we tried to do and the hurdles that we've overcome in that process. And, you know, I think that the working group is a great place to try to vet out some of those ideas and see what is really universal and reusable and what really isn't. Benji, how did you end up joining the working group? Yeah, good question. So I think I was working on a project at the time, PostGraphQL it was called, and I was just a contributor to it. I was a user of that piece of software. And as I started using it more, I got more involved in the maintenance of it. I've since taken over as the maintainer of it. And whilst working on that, there was things that I'd spotted in graphql itself in the spec that I felt, you know, could be enhanced. And I wanted to get involved. And what I really liked about the graphql working group is just how approachable it was. Like, there are very good guidelines on how to be involved. Like one of them is be involved. So like, if you're going to turn up to the graphql working group, like, plan to actually engage in the discussion. One of the easiest ways to get involved if you really just wanted to be a fly on the wall, which is where I went to get my initial confidence, as it were, was just to help out by taking notes. So that's, I think, how I got started. I turned up, I started taking notes, and as time went on, I started getting more and more involved in the discussions. Now I've had various pull requests merged into the graphql spec. I've got others that I'm working on. And I'm still there every month taking the notes as well. Yeah, it sounds like for me, it sounds like a cry for the community for someone else to maybe at least at some point, help Benji and take also some notes. So there's a lot of help we need. Maybe the first of top of the list is someone to join in and take notes. I'll say Benji undersells himself here. Not only is he one of the most significant contributors to the spec, the most value he provides is the notes that he basically co-runs our meetings. And that's not to undersell the value of the actual spec contributions he's made. The process itself running smoothly is super important. Yeah, definitely. Sasha, how was your experience joining? Yeah, I think I ended up joining when I was back working at Medium when we were first moving to graphql and just kind of wanted to learn more. And we had seen some internal issues that were like, maybe we could solve these and kind of started getting involved in the working group. And yeah, I mean, what everyone's saying is true. The community is super awesome. So joining the working group was just, everyone's really friendly and welcoming. And it was really cool to be able to have a space to kind of learn from the graphql community at large. Everyone's little companies were all kind of doing their own thing. And it's really nice to come out and see, well, what are other people doing? Or what are other people thinking about? And so that was really cool to be able to join and see that. And the other cool thing was that you can, one, you can just kind of draft an RFC and put it out there and see what people think. And a lot of times people are also thinking about the same thing. I can't count the number of times where I've seen an RFC and I was also thinking about that thing or I made one and other people were like, oh, yes, we need to solve this problem. So it's a really cool way to kind of share ideas and get things rolling and make really cool changes. Yeah, I feel the same. I feel like for me, at the beginning when I just started, I felt like there's some people somewhere making decisions and I have no saying and they don't know me and I don't know them and everything. And then the more I got into this, the more I realized it's completely open. And the more you could join, everything is recorded, you could learn. And most of all, the more you share, just like you said, the more you share your opinions and the more you share your questions, you realize not only that you might realize that your voice is being heard and you can actually affect things, but you can also learn a lot. Mostly there's people with so much experience and the discussions are so much in depth that it's mind blowing and it teaches you so much. I wonder, I don't know, what do you think would be the first thing, if someone wants to get involved and wants to get more deep into the drug, what would be the first step for people to join in there? Maybe Benji can take that. Sure. So, yeah, I'm often trying to encourage people to get involved in the graphql working group. So often people will engage with, for example, the graphql spec issues or pull requests. They'll raise up this feature that they'd like to see or this change that they'd like to see. And basically what I advise from everyone really is add yourself to the next agenda. We have agenda files in the graphql working group GitHub repository, and all you need to do is just add your name, your organization to that list. And you will also need to agree to the agreements that we have that are part of the graphql Foundation, but it's just a couple of sides of A4, sign it and you're done. If you're under an organization like Twitter, for example, that already have an agreement, then if you're one of the employees there, you just join it under that. So it's quite easy if you're part of one of those orgs. But even as an individual contributor like myself, very easy to do that. The bot checks for you to make sure it's been done, gives you a green tick when it's done, and then that's it. You can attend. And then you just turn up and you get involved. I think it's good to have an idea what you want to be involved in discussing. So maybe you've got your own item. But sometimes it's that there is something that's going to be discussed, like client controlled nullability or something like that, and you think, oh, this is relevant to my use case and I want to make sure that this is implemented in a way that works for me and for my project. And so, yeah, just turning up to be engaged in someone else's projects is another great thing to do. I'll also add that I don't think we mentioned this yet. We record all of these. And so the artifacts we show up in a couple of places. One, of course, Benji's high quality notes. He's not the only one taking notes. Other people take notes. But those are a great way to review what's happened. Also, the entire meeting gets posted to YouTube. We have a YouTube channel and it's literally automated. As soon as our meeting ends, Zoom does its thing in the background and a couple hours later it pops up on YouTube. So if you ever want to get a sense of what that meeting is like or follow up and see what was discussed, you can absolutely do that. One of our principles is to operate with as much transparency as possible. That's one of the reasons why we do that. But if there's something where you do want to get involved with a discussion, what Benji said is exactly right. All are welcome. I know a handful of people who they watch the graphql working group repo just so they can see, you know, basically the major changes that happen there is someone has an idea, it shows up as a file or someone wants to add something to agenda, it shows up as a pull request. And if you watch that repo, then you just get a sense of what's going on and you can say, oh, that's that that piques my interest. I would like to join this one and join that conversation. I'll also add that in the graphql.org website now, there's like a whole revamped community section that was being done by Brian from the foundation. And there, I think you can find all the links that we're all talking about here. So it's like a good one place where you can go and find the YouTube channel, the GitHub repos and everything. Yeah. So that's a good place to start. I wonder, I wonder what would be like the most like for people that are now want to join or just maybe we can hear from each one of you. What is the most exciting thing that are happening now in the working group? Just to give a sense of, you know, what's happening and to get people excited to join. I want to go first, if that's okay. Because I'm super excited about the client controlled nullability. That's like the raddest thing. There's a ton of rad things that are happening. But like most recently, I've been like super excited about that, because this is one of those things that we were talking about internally at Twitter. And like, I feel like a lot of other companies are kind of running into this, like a lot of places are like far enough along where you're at the point that like, wait a second, this is kind of getting annoying that the like, we have the server sending something back and we're like getting these responses that like, you know, the client doesn't know what to do with it, because like, actually, I did mean this thing that was required for me or like this thing actually can be no, I don't care. And it kind of really fits with like, sort of a graphql story, which is the client gets to decide what it needs and what it wants and like how it does things. So the client controlled nullability is like the perfect example of this and having like an operator to do that, have it all built into graphql would be like the raddest thing. So I'm super excited about that. And I think like, it would honestly change like a bunch of stuff at Twitter and make it like way, way nicer. Because right now, we're just like running into issues with that, where it's like we know the client really does need to be able to control that. And like, we can't do that right now. And it's actually kind of annoying. So I'm super excited about I think that's like the raddest thing. Benji, anything you're super interested in or excited about? There's so many things that are coming through that I'm pretty excited about. I think I'll mention one that I don't think anyone else is going to mention, which is the graphql fragment arguments. I think that's particularly interesting. One of the things that I really love about graphql is fragments themselves. I think you really need to use fragments quite a lot to do graphql, particularly for like react or client side applications to do it in a nice way. You really want to specify what data requirements a particular component has right next to that component. And fragments are a great way to do that. But at the moment, the arguments, sorry, the variables that you would use in your graphql document are kind of global to the document. So if you want to reference like, I don't know, an avatar size variable, that would be sort of shared by everything. So it's kind of like it's not local anymore. And one of the interesting things is this concept that I think Matt Mahoney from Facebook is the champion for this, is to basically give these fragments the ability to add arguments in the same way that your graphql operation itself can accept variables. Then when you use the fragment, you can specify variables at that level, whether they be constants like argument size 17, or whether they be like referencing a variable from higher up or something like that. And obviously it's still under discussion what it's actually going to be. But yeah, I think that's quite exciting and sort of helps this locality be even more powerful. Any other things that you want to, anyone wants to mention around cool new things that are happening? Yeah, I wanted to let a couple other folks go. And I'm glad Sasha jumped in with such passion about the client controlled nullability, because I'm super excited about that one. You know, a lot of the things that I'm most excited about are ones that we've been working on for a really, really, really long time. And I think this is one of the things that surprises people sometimes. They often have a negative impression at first with like how slow we move, but we move slow on purpose. You know, our goal here is to end up with a graphql spec and technology that lasts. And so we want to make decisions that are the right ones. And so we're more willing to make slow decisions that are right, rather than fast decisions that we have to change around a lot, because we understand change is super painful with the technology that's used as broadly as graphql. So for client controlled nullability, I had the first conversation about that idea in maybe 2014 or 2015 with one of the members of the relay team at the time. And at the time, we was like, this is an interesting idea, but like, no, like it doesn't make sense. We're not going to do it. Like it's antithetical to some of the principles of our server. It bubbled up a couple other times throughout the course of the working group, even, but the approach was never quite right. And I think part of why I'm so excited about it now is two things have happened. One is we've come to a consensus that this is a problem we're solving with very clear information about why, which has dramatically helped us narrow in on the right way to solve it. And the second part is that the actual proposal this time is actually converging on something that nearly everyone in these meetings agree is the right direction. There's still unanswered questions, of course, but that's part of the reason why I'm so excited about it. I'd be remiss if I didn't talk about defer and stream. I think for folks who have watched my talks for a while, maybe remember me giving a conference talk in 2016 or 2017 about defer and stream. And even at that point, we were talking about a fairly well-baked experiment that was internal to Facebook, which failed by the way. Like it didn't work. We rolled it back and we like gave that talk. I think Yuri, you shared the stage with me on that particular talk, if I remember correctly. Again, we work on these things for a long time and we want to finally get them right. And that one also came from a conversation with the relay team. And yesterday was the react Conf. And if anyone here tuned into that, there was a speaker from the relay team who talked a little bit about a directive that they use that is essentially client-controlled nullability and a directive that they use, which is essentially defer and stream. And these are all sort of like first principle ways that you build high quality clients that you integrate with react suspense. And so regardless of whether you're using react or not, like I think we got to appreciate just how much innovation comes out of that relay team, that a lot of the ideas that they try back propagate into the spec for anyone, regardless of platform to use. So I'm excited about defer and stream because of the new capability. I'm excited about it because we started having the conversation around those things in 2016 and they're finally now reaching reification. But also I have to give massive, massive props to the champions of these efforts. And I want to highlight defer and stream in particular because the folks at First Dibs, which is a company that's sponsoring this work, Rob in particular, Rob and Lillian over at that company have put in a massive amount of work to work through every edge and corner case, pushing experiments, pushing pull requests onto our reference implementations, working through spec techs for literally over a year. And the sponsoring of that work is the reason why it's going to make its way into the spec. And so like both for that company for sponsoring and to those humans for doing the work, I have massive gratitude. So I wanted to make sure that I mentioned that one. I think it's really interesting, right? Because when you take something like graphql, which is like, it helps so many of us do the things that we want to do across all of our work environments and the streams that we work through and everything that we're doing. And we often find local things that bother us from time to time, like I mentioned before, that are specific to the companies and the problems that we're working on. But the working group really allows us to kind of come together and have this overall kind of this works, this doesn't, and just a little bit more of a tweak here will get us into the right mode. And I think like you said, Lee, like sometimes people provide, you know, they bring up the topic of something that needs to be fixed, but it isn't just quite right yet. And once those things finally start to converge, but they've been discussed a lot and over and over again, and even prototyped in different platforms for different needs, we start to see how everybody can benefit from it. And we can narrow in on that really, really good solve and solution for everything that we that we want. And I think this kind of a thing, like if everybody wanted to go through to the, you know, to graphql and just modify it for their own purposes, it would very quickly cause problems amongst everybody. So we have to be careful, I think, in the big term about how does, how do these changes affect people and what does it do to all of those, you know, hundreds of thousands or millions of implementations that people have put together? And do we not break all of those things going backwards? And if we are going to break something, let's be very, very clear and make sure we know what that's going to do to everybody that's out there. Yeah, I agree. I mean, for me personally, I feel like starting to join and being exposed to this, to the working group basically moved me up a level in terms of how I see myself actually as a developer or as an engineer. Like I know it's going to be like a big sentence or something like that, but for me looking at how the working group evolves graphql taught me a lot about generally how to make change. Because there's a huge community, there's a lot of efforts, there's so many people who are invested in it, so many opinions, so many, you know, implementations, so many different incentives and things like that, and somehow the working group is able to, you know, find the right balance. And I think the more I, you know, get involved, the more I'm impressed by how well it is managed and how many opportunities it had to basically, we had to fuck up this thing. And it's not, and it's really, really impressive. So just, I think for me personally, just even to be a fly on the wall was really an amazing experience and still today I think it's very, very interesting, not only how to make change to graphql, but also how the working group operates itself. I think, you know, it's constantly changed and getting better. I think it's an amazing group and community effort that is being led in a very well way. So yeah, thanks for everyone who's involved in it and actually like making it like that. I think it's, you know, it's not obvious, but it's like that. I have another, maybe weird question or maybe like a niche question, but you know, there's also the graphql Foundation. How is, are those two related? Like there's the graphql working group, there's the graphql steering committee and there's the graphql Foundation. How are they related to each other? Yeah, I'll cover this one because there's some nuance here. And actually I think this is a really important question because the most common misconception that I get is that you can only contribute to graphql or you can only show up to one of these meetings if you're a quote member of the foundation or vice versa, that like showing up to these meetings somehow makes you part of the foundation. These are actually like two parallel things. So the graphql Foundation was formed in early 2019. And up until that point, graphql was a open source project sponsored by Facebook, much in the same way that react is still today. And you know, I think there's like different constraints on these projects. They're used in fairly different ways and especially graphql being a specification that is implemented in many different languages is a meaningfully different thing that requires a different operating model and actually requires a different IP model, like a different way that you actually protect the ideas themselves. There's also like a lot of the ideas for the foundation were born out of controversy. We are Facebook's early open source framing was I think novel, but not well received. And so like the licensing for things changed. I actually hold a patent to some of the core ideas of graphql that I wrote well before we ever thought it would be an open source project. And of course we've like offered a free license to everyone who uses it. But we've also looked at the history of how similar projects have worked. And I know many of the folks watching this use and love Node, go back and look at IOJS and steep yourself in the pain that went on in that community through that. I think that the health of the graphql community is not a natural conclusion. A lot of work goes into making sure that this community stays well aligned and there are many counter examples. So I had a lot of these things in mind when I set the foundation up and the foundation does a couple of really important things to talk about. One is that it actually gives us the legal framing for all of this. I come from, I was building software. I'm not a lawyer. So I had to steep myself in this much more than I would ever care to. But now I know way more about open source law than I ever did before. And we now have a very firm foundation that's based in precedent from a lot of other important projects. I built this thing in partnership with the Linux Foundation. They know their stuff. We didn't just cargo call things from previous projects. We used their lessons learned. I have a program manager, Brian, that we work super closely with. He's helped manage many super important projects, including the OpenJS Foundation, and has brought all of his experience to bear to help us set up a super firm foundation to make sure that everyone who contributes, contributes in good faith and everything that they do is then shareable to any company that wants to use it. That's really, really important to get right. Then there's the financing side. And you might not think too much about this, but, and of course, a lot of the folks here are volunteers and that's great, but it requires a lot to keep things operating well. So the graphql Foundation is actually formed from a bunch of companies. Companies join the foundation and they contribute finances to that. We have a foundation board. They get to direct where those finances go. And actually one of the most important ways that we use that pool of finances is a grant program. So we actually launched this grant program just very recently, a couple of weeks ago. You can find it at graphql.org slash community dash grant. I think there's a link in the bottom of the website in the footer there, where we give out fairly significant grants to people in the community who are doing important community focused work. So that's the graphql Foundation. It is a member based thing. Members pay to be part of that. And in doing so, they get to control how that pool of budget gets spent. In parallel to that is the graphql Technical Project. The graphql Technical Project is the spec itself, the reference implementation, the graphical tool, and a handful of other things. The foundation has a framing that creates the legal structure for those things. They own the IP to those things, but the board of that graphql Foundation does not get to make technical decisions. They formally delegated all technical decisions to the technical steering committee, which is this fine set of folks that you see here. We're the ones who get to make technical calls. That's a really important firewall between these two to make sure that the set of companies who may have business interest in bending graphql in a particular way, don't actually have the ability to do that. Maybe there's someone who happens to work at one of those companies who's on the steering committee, but they tend to be well-reasoned members. And the technical steering committee is actually a community voted in thing. The foundation doesn't even get to choose who that technical steering committee is. We get to self-select. And that's really important for doing that. So the technical steering committee, this group of folks is essentially the subset of the people who show up to these working group meetings who have explicit voting control. If there's ever a case where we need to make a firm decision and we want to have a counted vote, that's really the first and foremost thing that the technical steering committee is responsible for. The other is that we're kind of the cheerleaders or the champions of the group. If anything important needs to happen, there's like, who's going to do it? It tends to fall on one of the members of the steering committee. And then there's the working group. And we've spent a lot of time talking about that. That's an open invite. Anyone can come, anyone in the community who has an idea. The technical steering committee is the sponsor and champion of that meeting. They help operate it. Benji and I in particular tend to co-run those meetings, but we tend to delegate bits and pieces of that to this group as a whole. So hopefully that gives folks a much better sense of the details of the world, the finance half and the technical half. Yeah, exactly. I think your explanation now just proved my other point, which is how much thought is coming into creating an open community that stays open and keeps progressing. So thanks so much for creating all of this. Anything that any of you want to say just before we leave, we don't have a lot of time to continue this discussion. Any last words that you think is important for people to hear about the working group in general? I just want to say a huge thank you to everyone who's involved in the working group. There's a lot of, as Lee says, volunteer time that goes into advancing these things. And it's not just the case of raising a pull request and hoping it gets merged. It is factoring in all of the feedback from various other people into that. And also, I really like that we don't hold really tight to our initial ideas. We factor in that feedback and it goes forward. And I think that's really great, but it takes a lot of effort. So thank you. Huge thank you to everyone involved. Thanks. And thanks for everyone here in the final discussion. Thanks for participating. Thanks for your general work at the working group. It's amazing. So thank you so much for joining me and thanks for everyone who joined online. And I hope to see you all, or at least some of you, in the next working groups. Thank you so much. And in the chat. Oh yeah. We are all going to continue in the chat. So you can join us and ask us questions there.