React Advanced

React Advanced Conference 2022

Oliver Barnwell

<p>The Great British Public Toilet Map is an open source, community driven project dedicated to helping people find toilets across the UK, with around 14,000 loos recorded and counting. In 2021 we took on the challenge of migrating the project from a SPA React app written in JavaScript to NextJS and Typescript. Together we'll discover why we decided it was time to migrate, the myriad technical challenges we faced along the way, how this work benefits our users, and the many exciting plans we have for the future.</p><p> by <a href="https://www.froala.com/wysiwyg-editor?pb=1"> </a></p>

Making a Splash: The Story of a Toilet Map Migration

Hello, my name is Oliver and I'm here today to tell you about an interesting story that's taken place over the last year or so, probably a little bit more, and it involves a migration from Create React app to Next.js. The project in hand is the toilet map. It was previously known as the Great British Public Toilet Map, but we have plans in the future to maybe take it international. So it's quite an exciting project. I've been involved with it for, I think, around two years total in varying degrees of involvement. But over the last year, like I said, this has been quite a concentrated project to get this project fully modernised. So I'm going to tell you a story, essentially, why we did it. We're going to look at some of the interesting highlights from that journey. And I'll also cover for a brief moment why it's important that we're doing this work. So without further ado, let's dive straight in. So here's the logo and the URL, if you fancy visiting. And this is what it looked like in the first iteration. So this would have been around 2010, 2012, way before my first involvement in the project. And it was built, I think, using jQuery, pretty antiquated tech by today's standards. But it was already doing its job. And I think it only covered London boroughs. But very quickly, these things came along. And it was ordered to mobile. So we started to have more responsive layouts. And in its latest iteration, it's a bit like this. So we really put effort into making it responsive and easy for people to use, especially when they're on the go. So what is the state of public toilet provision in the United Kingdom right now? Unfortunately, it's not great. The person who founded the project, Gail Ramster, she's a senior research associate at the RCA, is quite critical of the state of public toilets in the UK. It's not a great situation. And from that quote, you can probably tell, for a fairly large portion of people, this is quite a big deal. Just before I was putting these slides together, I was browsing the web for some other quotes around this. This one particularly stood out to me. That's in eight years, we've lost over 10% of the public toilets in the UK, which is a pretty big deal. This is covered on various news outlets. And it really, well, while it is covered, there could probably be a lot more work put into helping this situation. And that as a project, that's what we're really trying to do. We're trying to sort the situation out in as best way as we can, given that we don't actually help to maintain toilets in the UK. Yep, this is what they look like. It's a pretty bad situation. Fortunately, there's another type of toilet that I haven't mentioned yet, known as publicly accessible toilets. These are provided by all kinds of businesses. And these are what we look to include in the map. So it's not just public toilets. It's toilets from a range of businesses, and anyone can contribute. On top of that, we also have community access schemes. So these businesses, they can sign up through various schemes run by local councils to register so that the people who really need toilet can look on the several data sets, but also on our map, where we've taken this data and plotted it so you can easily find out where you can go locally. So what does this data look like? We've got about 14,000 loos which have been marked as active. And over time, we've seen around 2,000 of these removed or updated. The open data for this, in part, is provided by the government and various other sources over the years. But user contributions, especially in the fairly recent past, have been the main way that we've expanded our data set. People contribute by going to this page on the website. And we have a nice little dialogue there where you can enter all the information around the toilet. We also have this Explorer tool so you can check out the data set, see what's been updated recently, and go back to the very start. So you can see back in 2014 when this was first put together. Briefly during lockdown, there was an amazing effort by an organisation called Locations UK to do a pretty similar thing to what we do. So they got really active, gained a lot of traction on social media, and loads of people contributed. It was an amazing effort. And after about, I think, about a year of activity, they decided that they would merge with us. So we've been given their data and we've got a really interesting technical challenge on our hands to take their data and integrate it with what we have on the project. It's definitely not simple because some of these locations could contain duplicates, some of them aren't very accurate. It's an interesting set of problems. So let's get onto the migration itself. What did it look like from a technical perspective? Well, as I mentioned, the project was a Create React app, so very client-side, JavaScript-heavy, no server-side rendering. And myself and Rupert, my collaborator on the project, took a look at what was available back in 2021 in terms of frameworks that we could move to. And next stood out. The features really speak for themselves. And as listed here, the ones that really stood out to us were around the server-side rendering and the ability to support root prefetching was especially interesting because we've got loads of markers on a map. It would be great to be able to prefetch some of these markers with a framework like this. So yeah, we started around here. So we thought, okay, trend's starting to go up. That's probably quite a good thing. Let's give it a go. This is what we went from. So we managed to migrate over to TypeScript during the course of the project. And we took full advantage of the fact that TypeScript is built with incremental adoption in mind. We also managed to achieve far better UX. We made it so that toilets loaded faster. We grouped the markers so it was easier to see them on the map. And we started caching the pages using some of the amazing features that Next offers. Also, for developers, anyone wishing to contribute to the project now will find it a far, far easier experience because we've adopted a language which is possibly one of the most popular on the planet these days. Also, the server-side rendering. So you don't see that anymore. You don't have JavaScript for some reason. You see that. Things will load a little bit faster because you're not having to pull a massive JavaScript bundle and execute it. Instead, you're just getting the HTML for the page. So let's take a whistle-stop tour of some of the things that we had to consider when we were doing that migration. Links, mainly a copy-paste job. Not too bad. Images also pretty copy-pastable. We also got to take advantage of some of the features that Next offers around images. So they handle a lot of the stuff that we would have had to cope by hand, like responsive image sizing. And the one that really interested me was around placeholders. Could we load in a blurred version of the image or other kind of optimizations? Because at the end of the day, what's driving our work here is to help people find a loop. Dynamic pages, pretty interesting feature. Here's just a very small snippet of how we generate one of these pages. We get a connection to the database and we render the page. So by the end of that method, you should have a fully rendered Lou detail panel for your mobile device. In terms of moving the API over, it was originally offered using the Express framework. And luckily, Next makes it super simple to take something like that over across to their platform. In terms of GraphQL, which we use as a front on top of our Mongo database, most of it was left largely unchanged. So this is what the architecture looks like now. Where the GraphQL plus Next.js part lives would have been the Express server. But now Next has kind of subsumed that Create React app and we've managed to put it all on the same level of the stack. We've also added this really nice little addition of a Redis cache layer, which means that when a user requests a map tile, we first check in our cache. And if it's there, we will give them that instead. This is what a Lou looks like. It's kind of evolved over many years. So the shape of it isn't perfect, but it does the job. And very quickly, this is how you grab a Lou from our GraphQL resolver. And in terms of running the GraphQL server, we connect to our database, you start Apollo server, and whoopsie, you have your API route. In terms of authentication, we were originally using Auth0, and we still are. In fact, Next makes it a dream to handle. They provide a high order component and we could just wrap one of our routes with that and everything worked as expected. So another key part of this work was improving the overall user experience of the project. So that was all around actually the mechanics of moving from Create React app to Next. And these are some things that we did on top of that to really just make the site better for everyone. The first thing we did was we took the size of an object for an individual Lou down from a pretty big blob of JSON, around 260 bytes, into a single string. And we're very quickly going to look at the anatomy of that string and how we're able to represent pretty much exactly the same information in a small amount of space. So if we break that string down, you'll see that it's made up of the Lou's ID, which is just unique, a geohash, and a filter. Before I started this, I had no idea what a geohash was and we'll dive into that briefly. But first of all, we're going to have a look at how we optimize these filters. So as a user of the map, you have these switches and they allow you to filter by various criteria. It's kind of a useful feature. These are easily represented using just bits, part of a binary number I'm sure you're familiar with. And that was quite a good realization because it meant that all of a sudden we could represent the state of these switches using those bits. And that's important because we went from a fairly large piece of information to just a single number to represent any combination of those filters we wanted to. So in this demonstration, we've got the very first bit highlighted, which means it's a no payment toilet. And the fifth one is highlighted, meaning that it's also got baby changing available. So we run a or operation on both of those two options. And the results of that operation is the number 17 or a binary word with those two bits set. And then once that's been sent to the client side, all we have to do is take the mask and do an and operation against the first and the fifth bits to determine whether they're on or off. So as you can see, we've run through each of the bits and we've determined that it's free and it has baby changing. So that's how we made the filters a bit better. Let's look at what GeoHash is. Now, I find these really exciting. But first, we're going to talk about what three words very quickly. It's a product you might have heard of. And it's a way of representing a location using, you guessed it, just three words. This is kind of similar in a way to that, but different in a very important sense. And how is it different? So this is a pretty concise description of what it does, I think, from the creator. Can't quite remember. But the key thing here is that by changing the length of our hash, we can make it more precise. So if it's only one character, it will represent a pretty wide chunk of the world. But as you increase precision on those characters, that very clever algorithm, which I couldn't start to understand, pieces together those characters into an increasingly precise representation of a location on Earth. So this is what that string represents in London, a little chunk of land right next to Trafalgar Square. So that was a pretty amazing realization for us. It meant that we could take a pretty long latitude and longitude and turn them into a consistently sized string of pretty good precision. It's enough for being able to find the toilet that's listed. And of course, with any compression scheme, you could always take it even further. So the ID is still quite long. We could probably turn this into a sequence of numbers. So we just sequentially refer to each loo or some other kind of mapping operation. But life is short, and we did a pretty good job. In terms of the fixed length of it, we could have just got rid of those pipes because we know the length of this string. We don't really need to separate it. That was really just for development. So that was all pretty exciting. I really enjoyed working on that. That was, if anything, just satisfying being able to get the amount of data down. And on the technical side, pretty critical because it meant that all of a sudden we're sending far, far less information to the client. And what this enables us to do is go from something that looked like this, where we were loading in everything in a pretty jumbled mess to a system of clustering. So instead of that, we employed a library called Marker Cluster, aptly named. And it lets us chuck our existing markers into a group, render the group, and magically they suddenly look like this, which is far better if you want to get a picture of a wider area, say if you're going holiday somewhere. The problem with that, even though it's great, is that clustering 13,000 markers is quite slow when you first load the page. It's a blocking operation. It could freeze the map on an older device for about five seconds, which isn't great at all. So we thought about a system of chunk loading. It's pretty simple in concept. We look at all the geohashes within the viewport. So these are some lower resolution geohashes which cover the whole of the UK. And within those boxes, we can request all of the toilets. So here we have the GraphQL operation. And here we have the code to actually get that information on the server side. I'm not going to step through it now. But if you're interested, it's all on GitHub. And as a result of that operation, you get a list of all the layers in the box. And then on the client side, we have some other React code which is used to load in the markers for each specific box. And to visualize it, it looks something like that. So each of those boxes is another instance of that component. And this is one of those auto-generated hooks which I mentioned, which is just generated from our GraphQL schema. And we can call it. Apollo will cache the result. Things are quite fast now. And in terms of drawing markers, we have two hooks which allow us to stop rendering markers when they leave the viewport. So we can only render what's on the screen. So now when you're using the map, it looks a bit like this. So behind the scenes, we increase the level of detail as you zoom in using a method like this. And as a quick demonstration, this is what it looks like for a user these days. Far better to use, much nicer experience, and hopefully helping a lot of people in need find a toilet where no one will go. So another exciting thing was accessibility overlay, which provided the ability for people who maybe couldn't see as well when they're using application to use a dialogue that appears over the map. It's just overlaid on the top. And it provides a list of the first five toilets within that box that you see in the middle. So as you pan around the map, even without being able to see, you can still access the data that we provide. Pretty important for accessibility. So we don't really add the name to the compressed string. So we just had to add another query which allows us to fetch the loon names on demand. So you hover over a bot, you move the bot, you find new names. This is what it looks like. So you can tab around with your keys and access all the same data that anyone else could. In terms of testing and GitHub actions, this was pretty new to the project. And we moved essentially from no continuous integration apart from some very, very broken end to end tests to a very comprehensive suite of Cypress tests, which has been pretty incredible for us as contributors because it means we can move faster and with more confidence, especially as it's just the two of us. So very quickly, I want to tell you all about the future. What else do we have in store for this project? There's quite a lot, if I'm being completely honest. The thing about a product like this is everyone needs to use the toilet. Everyone has an opinion on what could be changed. Everyone wants to kind of immediately relate to the kind of problems that we're looking to solve. I've never had a conversation with somebody who has gone and not had a suggestion for a new feature. And we keep track of these on GitHub. So we make issues whenever something comes through. We also actively gather user feedback. So we have a little user feedback box on the website. And all of the time, people just put their thoughts as they're using the app and give us an idea of what could be improved. And often it's stuff which we would have never spotted ourselves. On top of that, it's only limited to the UK right now. There are some toilets mapped in other countries, but it's definitely not officially supported by the project. So a massively interesting piece of work that we have coming up and we'd love for help with is around the question of how do we take this to other countries? Do we need to think about internationalization problems? How do we store the data? Right now we run on a very limited budget of around 40 pounds a month. How do we allow someone who wants to start their own version of the map in France to pay for that infrastructure? Would it be syndicated, perhaps? These are all pretty cool questions which I could spend another hour talking about. But I'm going to draw the talk to a close with this little 3D printed toilet map logo I made for Rupert for Christmas. I thought it would be a lovely little present for him. And a very quick shout out to everyone else who's contributed to the project over the years. It's been a real group effort. And it's definitely still ongoing. And with all this work that we've been putting into it, we really hope that we'll get more contributions from people who are interested, whether from the tech perspective, user experience side of things, or just from an advocacy point of view. If you're interested in helping people find a toilet, this project is for you. Thank you so much for listening. I hope you learned something. And if anything, I'm very glad that I could spread awareness of the project because I believe it's doing real social good. Thank you.
