Making a Splash: The Story of a Toilet Map Migration

Rate this content
Bookmark

The Great British Public Toilet Map is an open source, community driven project dedicated to helping people find toilets across the UK, with around 14,000 loos recorded and counting. In 2021 we took on the challenge of migrating the project from a SPA React app written in JavaScript to NextJS and Typescript. Together we'll discover why we decided it was time to migrate, the myriad technical challenges we faced along the way, how this work benefits our users, and the many exciting plans we have for the future.

24 min
24 Oct, 2022

AI Generated Video Summary

The Talk discusses the migration of the Toilet Map Project from Create React app to Next.js, with plans to take it international. The project aims to address the inadequate public toilet provision in the UK. It includes publicly accessible toilets, community access schemes, and user contributions. The migration to Next.js offers benefits like server-side rendering and root prefetching. The Talk also covers the migration to TypeScript, improvements in loading speed and caching, dynamic pages, API migration, geohashing, marker clustering, chunk loading, accessibility overlay, and future plans for expansion and user contributions.

1. Introduction to the Toilet Map Project

Short description:

Hello, my name is Oliver, and I'm here today to tell you about a migration from Create React app to Next.js. The project in hand is the toilet map. It was previously known as the Great British Public Toilet Map. But we have plans to take it international. I'll cover why it's important that we're doing this work. Here's the logo and the URL. The state of public toilet provision in the UK is not great. We've lost over 10% of the public toilets in the UK in eight years. As a project, we're trying to sort the situation out.

Hello, my name is Oliver, and I'm here today to tell you about an interesting story that's taken place over the last year or so, probably a little bit more. And it involves a migration from Create React app to Next.js.

The project in hand is the toilet map. It was previously known as the Great British Public Toilet Map. But we have plans in the future to maybe take it international. So it's quite an exciting project. I've been involved with it for I think around two years total in varying degrees of involvement. But over the last year, like I said, this has been quite a concentrated project to get this project fully modernised.

So I'm going to tell you a story, essentially why we did it. We're going to look at some of the interesting highlights from that journey. And I'll also cover for a brief moment why it's important that we're doing this work. So without further ado, let's dive straight in.

So here's the logo and the URL, if you fancy visiting. And this is what it looked like in the first iteration. So this would have been around 2010, 2012, way before my first involvement in the project. And it was built, I think, using jQuery, pretty antiquated tech by today's standards, but it was already doing its job. And I think it only covered London boroughs. But very quickly, these things came along, and it was ordered to mobile. So we started to have more responsive layouts. And in its latest iteration, it's a bit like this. So we really put effort into making it responsive and easy for people to use, especially when they're on the go.

What is the state of public toilet provision in the United Kingdom right now? Unfortunately, it's not great. The person who founded the projects, Gayle Ramster, she's a senior research associate at the RCA, is quite critical of the state of public toilets in the UK, it's not a great situation. And from that quote, you can probably tell, for a fairly large portion of people, this is quite a big deal. Just before I was putting these slides together, I was browsing the web for some other quotes around this. This one particularly stood out to me, that's in eight years, we've lost over 10% of the public toilets in the UK, which is a pretty big deal. This is covered on various news outlets. And it really, well, while it is covered, there could probably be a lot more work put into helping this situation. And as a project, that's what we're really trying to do. We're trying to sort the situation out in as best way as we can, given that we don't actually help to maintain toilets in the UK.

2. Public Toilets and Data Integration

Short description:

There are publicly accessible toilets provided by various businesses that we include in the map. We also have community access schemes where businesses can register. The data set consists of 14,000 active locations, with user contributions being the main source of expansion. During lockdown, we merged with Lucations UK and faced challenges in integrating their data. The migration to Next.js offered features like server-side rendering and root prefetching.

Yeah, this is what they look like. It's a pretty bad situation. Fortunately, there's another type of toilet that I haven't mentioned yet, known as publicly accessible toilets. These are provided by all kinds of businesses. And these are what we look to include in the map. So it's not just public toilets. It's toilets from a range of businesses and anyone can contribute.

On top of that, we also have community access schemes. So these businesses, they can sign up through schemes run by local councils to register so that the people who really need toilet can look on several data sets, but also on our map, where we've taken this data and plotted it. So you can easily find out where you can go locally.

So what does this data look like? We've got about 14,000 loopholes which have been marked as active and over time we've seen around 2,000 of these removed or updated. The open data for this in part is provided by the government and various other sources over the years. But user contributions, especially in the fairly recent past have been the main way that we've expanded our data set. People contribute by going to this page on the website and we have a nice little dialogue there where you can enter all the information around the toilet. We also have this explorer tool so you can check out the data set, see what's been updated recently, and go back to the very start. So you can see back in 2014 when this was first put together.

Briefly during lockdown, there was an amazing effort by an organisation called Lucations UK to do a pretty similar thing to what we do. So they got really active, gained a lot of traction on social media and loads of people contributed. It was an amazing effort and after about I think about a year of activity, they decided that they would merge with us. So we've been given their data and we've got a really interesting technical challenge on our hands to take their data and integrate it with what we have on the project. It's definitely not simple because some of these locations could contain duplicates, some of them aren't very accurate. It's an interesting set of problems.

So let's get on to the migration itself. What did it look like from a technical perspective? Well, as I mentioned, the project was a Create React app. So very client-side, JavaScript-heavy, no server-side rendering. And myself and Rupert, my collaborator on the project, took a look at what was available back in 2021 in terms of frameworks that we could move to and next stood out. The features really speak for themselves and as listed here, the ones that really stood out to us were around the server-side rendering and the ability to support root prefetching was especially interesting because we've got loads of markers on a map, it'd be great to be able to prefetch some of these markers with a framework like this. So, yeah, we started around here. So we thought, okay, trend's starting to go up, that's probably quite a good thing, let's give it a go. This is what we went from.

3. Migration to TypeScript and Next.js

Short description:

We migrated to TypeScript, improving UX, loading speed, and caching pages. Developers will find it easier to contribute. Server-side rendering improves loading time. Migration considerations included links, images, and image optimization.

So we managed to migrate over to TypeScript during the course of the project, and we took full advantage of the fact that TypeScript is built with incremental adoption in mind. We also managed to achieve far better UX, we made it so that toilets loaded faster, we grouped the markers so it was easier to see them on the map, and we started caching the pages using some of the amazing features that Next offers.

Also, for developers, anyone wishing to contribute to the project now will find it a far, far easier experience, because we've adopted a language which is possibly one of the most popular on the planet these days. Also, the server-side rendering, so you don't see that anymore, you don't have JavaScript for some reason, you see that now. Things will load a little bit faster because you're not having to pull a massive JavaScript bundle and execute it, instead you're just getting the HTML for the page.

So, let's take a whistle-stop tour of some of the things that we had to consider when we were doing that migration. Links, mainly a copy-paste job, not too bad. Images, also pretty copy-pastable. We also got to take advantage of some of the features that Next offers around images, so they handle a lot of the stuff that we would have had to cope by hand, like responsive image sizing, and the one that really interested me was around place holders. Could we load in a blurred version of the image or other kinds of optimizations, because, at the end of the day, what's driving our work here is to help people find a loop.

4. Dynamic Pages and API Migration

Short description:

Dynamic pages, moving the API over, GraphQL architecture, Redis cache layer, Aloo, running the GraphQL server, authentication, improving user experience, optimizing filters, representation using bits.

Dynamic pages, a pretty interesting feature. Here's just a very small snippet of how we generate one of these pages. We get a connection to the database, and we render the page. By the end of that method, you should have a fully rendered Lou detail panel for your mobile device.

In terms of moving the API over, it was originally offered using the express framework. Luckily, Next makes it super simple to take something like that over across to their platform. In terms of GraphQL, which we use as a front on top of our Mongo database, most of it was left largely unchanged, so this is what the architecture looks like now. Where the GraphQL plus the Next.js parts lives would have been the express server, but now Next has kind of subsumed that Create React app and we've managed to put it all on the same level stack. We've also added this nice little addition of a Redis cache layer, which means that when a user requests a map tile, we first check in our cache, and if it's there, we will give them that instead. This is what Aloo looks like. It's kind of evolved over many years, so the shape of it isn't perfect, but it does the job, and very quickly, this is how you grab Aloo from our GraphQL resolver, and in terms of running the GraphQL server, we connect to our database, we start Apollo server, and you have your API root.

In terms of authentication, we were originally using Auth0, and we still are. In fact, Next makes it a dream to handle. They provide a high order component, and we could wrap one of our routes with that, and everything worked as expected. So, another key part of this work was improving the overall user experience of the project. So, that was all around actually the mechanics of moving from create react app to Next. These are some things we did as on top of that to really just make the site better for everyone. The first thing we did was we took the size of an object for an individual loop down from a pretty big blob of JSON, around 260 bytes into a single string. And we're very quickly gonna look at the anatomy of that string and how we're able to represent pretty much exactly the same information in a small amount of space.

So, if we break that string down, you'll see that it's make up of the lose ID which is unique. A geohash and a filter. Before I started this, I had no idea what a geohash was, and we'll dive into that briefly. But first of all, we're gonna have a look at how we optimize these filters. So, as a user of the map, you have these switches, and they allow you to filter by various criteria. It's a kind of useful feature. These are easily represented using just bits. Part of a binary number I'm sure you're familiar with. And that was quite a good realization, because it meant that all of a sudden, we could represent the state of these switches using those bits. And that's important, because we went from a fairly large piece of information to just a single number to represent any combination of those filters we wanted to. So, in this demonstration, we've got the very first bit highlighted, which means it's a no payment toilet.

5. Improving Filters with Bit Operations

Short description:

We run an OR operation on the options and use a mask to determine the status of each bit. This improved the filters.

And the fifth one is highlighted, meaning that it's also got baby-changing available. So, we run an OR operation on both of those two options. And the result of that operation is the number 17, or a binary word with those two bits set. And then once that's been sent to the client side, all we have to do is take the mask and do an AND operation against the first and the fifth bits to determine whether they're on or off. So, as you can see, we've run through each of the bits and we've determined that it's free and it has baby-changing. So, that's how we made the filters a bit better.

6. Geohash and Marker Clustering

Short description:

Let's look at what a geohash is. It's a way of representing a location using a string of characters. By changing the length of the hash, we can make it more precise. This enables us to turn latitude and longitude into a consistently sized string. We employed a library called MarketBuster to cluster our markers and improve loading speed.

Let's look at what a geohash is. You know, I found these really exciting, but first we're going to talk about what3words very quickly. It's a product you might have heard of and it's a way of representing a location using, you guessed it, just three words. This is kind of similar in a way to that, but different in a very important sense.

And how is it different? So this is a pretty concise description of what it does. I think from the creator, I can't quite remember. But the key thing here is that by changing the length of our hash, we can make it more precise. So if it's only one character, it will represent a pretty wide chunk of the world. But as you increase precision on those characters, that very clever algorithm, which I couldn't start to understand, pieces together those characters into an increasingly precise representation of a location on Earth. So this is what that string represents in London. A little chunk of land, right next to Trafalgar Square.

So that was a pretty amazing realization for us. It meant that we could take a pretty long latitude and longitude and turn them into a consistently sized string of pretty good precision. It's enough for being able to find the toilet that's listed. And of course with any compression scheme you could always take it even further. So the ID is still quite long. We could probably turn this into a sequence of numbers. So we just sequentially refer to each loop or some other kind of mapping operation. But life is short and we did a pretty good job. In terms of the fixed length of it, we could have just got rid of those pipes because we know the length of this string. We don't really need to separate it. That was really just for development.

So that was all pretty exciting. I really enjoyed working on that. That was, if anything, just satisfying being able to get the amount of data down. And on the technical side, pretty critical, because it meant that all of a sudden we're sending far, far less information to the client. And what this enables us to do is go from something that looked like this, where we were loading in everything in a pretty jumbled mess, to a system of clustering. So instead of that, we employed a library called MarketBuster, aptly named. And it lets us chuck our existing markers into a group, render the group, and magically, they suddenly look like this, which is far better if you want to get a picture of a wider area, so if you're going holiday somewhere. The problem with that, even though it's great, is that clustering 13,000 markers is slow when you first load the page.

7. Chunk Loading and Accessibility Overlay

Short description:

We implemented a system of chunk loading to prevent the map from freezing on older devices. By requesting toilets within specific geohashes, we can load markers for each box and render them on the client side. With increased level of detail as you zoom in, the map now provides a better user experience. An accessibility overlay allows users with visual impairments to access toilet data by providing a list of toilets within the visible area. We also added a query to batch load location names on demand.

It's a blocking operation. It could freeze the map on an older device for about five seconds, which isn't great at all. So we thought about a system of chunk loading. It's pretty simple in concept. We look at all the geohashes within the viewport. These are some lower resolution geohashes, and within those boxes, we can request all of the toilets. So here we have the operation, and here we have the code to actually get that information on the server side. I'm not going to step through it now, but if you're interested, it's all on GitHub.

And as a result of that operation, you get a list of all the loads in the box. And then on the client side, we have some other React code, which is used to load in the markers for each specific box, and to visualise it, it looks something like that. So each of those boxes is another instance of that component. And this is one of those auto generated hooks, which I mentioned, which is just generated from our GraphQL schema, and we can call it Apollo will cache the result. Things are quite fast now. And in terms of during the markers, we have two hooks, which allow us to stop rendering markers when they leave the viewport, so we can only render what's on the screen.

So, now when you're using the map, it looks a bit like this. So, behind the scenes, we increase the level of detail as you zoom in, using a method like this. And as a demonstration, this is what it looks like for a user these days. Far better to use. Much nicer experience and hopefully helping a lot of people in need find a toilet where no one will go. So, another exciting thing was an accessibility overlay which provided the ability for people who maybe couldn't see as well when they're using application to use a dialogue that appears over the map. It's just overlaid on the top. And it provides a list of the first five toilets within that box that you see in the middle. So, as you pan around the map, even without being able to see, you can still access the data that we provide. Pretty important for accessibility. So, we don't really add the name to the compressed string. So, we just had to add another query which allows us to batch the loon names on demand. It will hover over a bot. You move the bot and find new names. This is what it looks like. So, you can tap around with your RPs.

8. Future Plans and Call for Contributions

Short description:

We implemented comprehensive Cypress tests for continuous integration. We actively gather user feedback and plan to expand the project to other countries. We have questions about internationalization, data storage, and infrastructure funding. The project is a group effort and welcomes contributions from various perspectives. Thank you for listening and supporting our mission.

And access all the same data that anyone else could. In terms of testing and GitHub actions, this was pretty new to the project. And we moved, essentially, from no continuous integration, apart from some very, very broken end to end tests, to a very comprehensive suite of Cypress tests, which has been pretty incredible for us as contributors, because it means we can move faster and with more confidence, especially as it's just the two of us.

So very quickly, I want to tell you all about the future. What else do we have in store for this project? There's quite a lot, if I'm being completely honest. The thing about a product like this is everyone needs to use the toilet, everyone has an opinion on what could be changed. Everyone wants, can immediately relate to the kind of problems that we're looking to solve. I've never had a conversation with somebody who has gone and not had a suggestion for a new feature. And we keep track of these on GitHub. So we make issues whenever something comes through.

We also actively gather user feedback. So we have a little user feedback box on the website. And all of the time, people will just put their thoughts as they're using the app and give us an idea of what could be improved. And often, it's stuff which we would have never have spotted ourselves. On top of that, it's only limited to the UK right now. There are some toilets mapped in other countries. But it's definitely not officially supported by the project. So a massively interesting piece of work that we have coming up. And we'd love for help with is around the question of how do we take this to other countries? Do we need to think about internationalization problems? How do we store the data? Right now, we run on a very limited budget of around 40 pounds a month. How do we allow someone who wants to start their own version of the map in France to pay for that infrastructure? Would it be syndicated perhaps? These are all pretty cool questions which I could spend another hour talking about.

But I'm going to draw the talk to a close with this little 3D printed toilet map logo I made for Rupert for Christmas. I thought it would be a lovely little present for him. And a very quick shout out to everyone else who's contributed to the projects over the years. It's been a real group effort. And it's definitely still ongoing. And with all this work that we've been putting into it, we really hope that we'll get more contributions from people who are interested whether from the tech perspective, user experience side of things, or just from an advocacy point of view if you're interested in helping people find a toilet, this project is for you. Thank you so much for listening. I hope you learned something. And if anything, I hope I'm very glad that I could spread awareness of the project because I believe it's doing real social good.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

React Advanced Conference 2022React Advanced Conference 2022
25 min
A Guide to React Rendering Behavior
React is a library for "rendering" UI from components, but many users find themselves confused about how React rendering actually works. What do terms like "rendering", "reconciliation", "Fibers", and "committing" actually mean? When do renders happen? How does Context affect rendering, and how do libraries like Redux cause updates? In this talk, we'll clear up the confusion and provide a solid foundation for understanding when, why, and how React renders. We'll look at: - What "rendering" actually is - How React queues renders and the standard rendering behavior - How keys and component types are used in rendering - Techniques for optimizing render performance - How context usage affects rendering behavior| - How external libraries tie into React rendering
React Summit Remote Edition 2021React Summit Remote Edition 2021
33 min
Building Better Websites with Remix
Remix is a new web framework from the creators of React Router that helps you build better, faster websites through a solid understanding of web fundamentals. Remix takes care of the heavy lifting like server rendering, code splitting, prefetching, and navigation and leaves you with the fun part: building something awesome!
React Advanced Conference 2022React Advanced Conference 2022
30 min
Using useEffect Effectively
Can useEffect affect your codebase negatively? From fetching data to fighting with imperative APIs, side effects are one of the biggest sources of frustration in web app development. And let’s be honest, putting everything in useEffect hooks doesn’t help much. In this talk, we'll demystify the useEffect hook and get a better understanding of when (and when not) to use it, as well as discover how declarative effects can make effect management more maintainable in even the most complex React apps.
React Summit 2022React Summit 2022
20 min
Routing in React 18 and Beyond
Concurrent React and Server Components are changing the way we think about routing, rendering, and fetching in web applications. Next.js recently shared part of its vision to help developers adopt these new React features and take advantage of the benefits they unlock.In this talk, we’ll explore the past, present and future of routing in front-end applications and discuss how new features in React and Next.js can help us architect more performant and feature-rich applications.
React Advanced Conference 2021React Advanced Conference 2021
27 min
(Easier) Interactive Data Visualization in React
If you’re building a dashboard, analytics platform, or any web app where you need to give your users insight into their data, you need beautiful, custom, interactive data visualizations in your React app. But building visualizations hand with a low-level library like D3 can be a huge headache, involving lots of wheel-reinventing. In this talk, we’ll see how data viz development can get so much easier thanks to tools like Plot, a high-level dataviz library for quick & easy charting, and Observable, a reactive dataviz prototyping environment, both from the creator of D3. Through live coding examples we’ll explore how React refs let us delegate DOM manipulation for our data visualizations, and how Observable’s embedding functionality lets us easily repurpose community-built visualizations for our own data & use cases. By the end of this talk we’ll know how to get a beautiful, customized, interactive data visualization into our apps with a fraction of the time & effort!

Workshops on related topic

React Summit 2023React Summit 2023
170 min
React Performance Debugging Masterclass
Featured WorkshopFree
Ivan’s first attempts at performance debugging were chaotic. He would see a slow interaction, try a random optimization, see that it didn't help, and keep trying other optimizations until he found the right one (or gave up).
Back then, Ivan didn’t know how to use performance devtools well. He would do a recording in Chrome DevTools or React Profiler, poke around it, try clicking random things, and then close it in frustration a few minutes later. Now, Ivan knows exactly where and what to look for. And in this workshop, Ivan will teach you that too.
Here’s how this is going to work. We’ll take a slow app → debug it (using tools like Chrome DevTools, React Profiler, and why-did-you-render) → pinpoint the bottleneck → and then repeat, several times more. We won’t talk about the solutions (in 90% of the cases, it’s just the ol’ regular useMemo() or memo()). But we’ll talk about everything that comes before – and learn how to analyze any React performance problem, step by step.
(Note: This workshop is best suited for engineers who are already familiar with how useMemo() and memo() work – but want to get better at using the performance tools around React. Also, we’ll be covering interaction performance, not load speed, so you won’t hear a word about Lighthouse 🤐)
React Advanced Conference 2021React Advanced Conference 2021
132 min
Concurrent Rendering Adventures in React 18
Featured WorkshopFree
With the release of React 18 we finally get the long awaited concurrent rendering. But how is that going to affect your application? What are the benefits of concurrent rendering in React? What do you need to do to switch to concurrent rendering when you upgrade to React 18? And what if you don’t want or can’t use concurrent rendering yet?

There are some behavior changes you need to be aware of! In this workshop we will cover all of those subjects and more.

Join me with your laptop in this interactive workshop. You will see how easy it is to switch to concurrent rendering in your React application. You will learn all about concurrent rendering, SuspenseList, the startTransition API and more.
React Summit Remote Edition 2021React Summit Remote Edition 2021
177 min
React Hooks Tips Only the Pros Know
Featured Workshop
The addition of the hooks API to React was quite a major change. Before hooks most components had to be class based. Now, with hooks, these are often much simpler functional components. Hooks can be really simple to use. Almost deceptively simple. Because there are still plenty of ways you can mess up with hooks. And it often turns out there are many ways where you can improve your components a better understanding of how each React hook can be used.You will learn all about the pros and cons of the various hooks. You will learn when to use useState() versus useReducer(). We will look at using useContext() efficiently. You will see when to use useLayoutEffect() and when useEffect() is better.
React Advanced Conference 2021React Advanced Conference 2021
174 min
React, TypeScript, and TDD
Featured WorkshopFree
ReactJS is wildly popular and thus wildly supported. TypeScript is increasingly popular, and thus increasingly supported.

The two together? Not as much. Given that they both change quickly, it's hard to find accurate learning materials.

React+TypeScript, with JetBrains IDEs? That three-part combination is the topic of this series. We'll show a little about a lot. Meaning, the key steps to getting productive, in the IDE, for React projects using TypeScript. Along the way we'll show test-driven development and emphasize tips-and-tricks in the IDE.
React Advanced Conference 2021React Advanced Conference 2021
145 min
Web3 Workshop - Building Your First Dapp
Featured WorkshopFree
In this workshop, you'll learn how to build your first full stack dapp on the Ethereum blockchain, reading and writing data to the network, and connecting a front end application to the contract you've deployed. By the end of the workshop, you'll understand how to set up a full stack development environment, run a local node, and interact with any smart contract using React, HardHat, and Ethers.js.
React Summit 2023React Summit 2023
151 min
Designing Effective Tests With React Testing Library
Featured Workshop
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents- The different kinds of React application tests, and where component tests fit in- A mental model for thinking about the inputs and outputs of the components you test- Options for selecting DOM elements to verify and interact with them- The value of mocks and why they shouldn’t be avoided- The challenges with asynchrony in RTL tests and how to handle them
Prerequisites- Familiarity with building applications with React- Basic experience writing automated tests with Jest or another unit testing framework- You do not need any experience with React Testing Library- Machine setup: Node LTS, Yarn