Service Workers: How to Run a Man-in-the-middle Attack on Your Own Site for Fun and Profit

♪♪ ♪♪ ♪♪ ♪♪ Welcome to Service Workers or how to run a man-in-the-middle attack on your own site for fun and profit. We, and by we I mean web developers, have broken the web. We've built the front end around JavaScript, which is a fragile house of cards. It's unreliable and easily broken, as anyone who's ever run into a blank web page or a button that does nothing when clicked can easily attest to. All of this JavaScript has big performance implications as well. Bandwidth has gone way up in the last five years. It's actually about three times faster on average on both mobile and desktop than it was in 2017. But because websites have gotten bigger as well and because so much of the front end is rendered in the browser with JavaScript now, the web isn't actually meaningfully faster than it was five years ago. And the problem with averages is that some countries have internet that's actually up to six times faster than five years ago. But many countries continue to struggle with desktop speeds that are slower than the average mobile speed was five years ago. Bandwidth is not evenly distributed. And as is often the case, people who live in poverty tend to suffer the most. So what I want to talk to you about today are service workers, a newer-ish tool in our toolkit that we can use to provide more resilience in the things that we build. Service workers can make our sites faster and allow us to build websites and apps that continue to function even when things go wrong. Hi, I'm Chris Ferdinandi. That's my face. You can find me online at GoMakeThings.com. I'm known on the internet as the Vanilla JS guy. I teach people JavaScript and, ironically, spend a lot of my time telling people to use less of it in the things that we build. I write a free daily newsletter and create courses and run workshops. And you can find more info about all that at GoMakeThings.com. Here's the agenda for today's talk. We're going to spend a bunch of time talking about what service workers are and how they work. And then we're going to dig into some specific strategies you can use when implementing them. Finally, we'll take a look at some cool things that you can do with them. I find that looking at specific, tangible examples helps make this stick. We are going to look at code, but we only have about 18 minutes left and you could easily fill an hour-long talk with just code examples, so we're going to stick to some pretty high-level surface examples. So, what is a service worker? Whenever a browser accesses a website or web app that you've built, it reaches out to the network and it gets a bunch of assets back. HTML, CSS, JavaScript, images, fonts. A service worker is a JavaScript file that your website installs into the browser that sits between the browser and the network. And to do that, wherever you would normally load the rest of your JavaScript, you write a little bit of JS. You check to see that the navigator object exists and that it has the service worker property, because older browsers don't support this. And if it does, you can use the register method to register a service worker, which is just a JavaScript file. And then in the background, the browser will download that file asynchronously and the next time a user visits your website, it will install it and activate it. And once it does, your service worker intercepts all requests that go out to the network and all responses that come back from it. And because a service worker is a JavaScript file, we can do that with a fetch event listener, just using the addEventListener method. And we can do things with those requests and responses. What makes service workers really powerful is that they have a built-in storage mechanism. They have a cache, and it can hold a lot of stuff, way more than local storage or cookies can. And you can actually take those responses that come back, save copies of them in your cache, and if something goes wrong with the network, you can load assets from your locally saved cache instead of the network, or cut it out altogether if you want. A service worker is a man-in-the-middle attack on your own website, but like a good one. Now, obviously, there's a lot of potential for abuse with something like this. So service workers require browser encryption and SSL certificate to work. There's an exception to this made for locally hosted sites. So if you're just running it on your laptop to test it, you don't need a certificate for that. But as soon as it goes out on the web and you visit it with a URL, you're going to need to install an SSL certificate, or the service worker won't work. Let's take a look at some service worker strategies. There are two basic approaches that you can use, network first and offline first. With a network first approach, you initially check the network to see if there are any responses. And if you get one back, you pass it through to the browser, basically saving a copy of it in your cache. If for some reason something goes wrong and it can't find that asset or your site goes offline, you can check the cache, your local storage, to see if you have a saved version of that request. And if you do, you can send that along instead. And here's what the code for that looks like. You're going to use the respondWith method on the event request. And for this approach, you actually use a vanilla JavaScript fetch method to take that request and make another call with that request. When you get a response back, you can use the clone method on the response to make a copy of it and save it to your cache. And then you can return the response back to the browser. And if something goes wrong inside your catch event on the fetch method, you can check the cache to see if you have a copy of that request already stored. And if you do, you can send the response associated with it back. It's really, really handy. Network-first works best for frequently updated assets. So HTML documents, if you have APIs that you're calling often and you always want to get the freshest data, you probably want to rely on a network-first approach for those as well. Offline-first works kind of the same way, but it just reverses the order. So when you get a request in, the first thing you do is check your local cache to see if you have a version of that request saved already. And if you do, you use that. If you don't, then you reach out to the network and you save a copy in your cache before sending it back to the browser. And here's what the code for that looks like. Once again, we're using the respondWith method to send a response back for that request. But this time, we're checking the app cache first to see if there's anything stored there that matches the request. If there is, we send it back. But if not, we use the fetch method again to make a live call to the network for that response, or for that request, rather. And when we get it back, we can clone it, save it into our cache, and then return the response. Offline-first is best for static assets that don't change as much. So CSS, JavaScript, images, fonts. You can also use service workers to provide fallbacks when things go wrong. And this works for both network-first and offline-first approaches. With a network-first approach, you make a call to the network. If you get nothing back, you check the cache. And if you still find nothing back, you can return a different asset. This could be something that you downloaded when your service worker installed for the first time. You may preemptively cache some fallbacks for when things go wrong. Or you could make a live call to the network in real time. With an offline-first approach, you do the same thing, but in reverse. You check your cache. You don't find anything. You check the network. And if you still don't find anything, or you can't reach the network, then you send along a fallback instead. And here's what the code for that might look like. When a service worker installs, it actually triggers an install event inside the service worker file itself. So you can listen for that with an event listener. And when that happens, you can open up a new cache and make a request to whatever assets you want to save with the new request constructor. And so in this example here, I'm requesting the offline HTML document that I want to send to people whenever they can't find pages. And then inside my fetch event listener, inside the catch handler, I am going to check to see if that request is cached. And if for some reason it's not, then I'm going to send back the offline HTML document that I have cached instead. Let's take a look at some uses and examples of service workers. This is where I think things really start to stick. So a really low-hanging fruit here, the example we actually just looked at, is showing critical information when a site goes offline. So if someone loses connectivity entirely, you can still give them a usable experience. And this is really particularly useful for things like restaurants, conferences, and hotels. With a restaurant, for example, you might let the user know they're offline and then give them other things that they might need or want from your restaurant, phone number, an address or directions on how to get there, maybe an abridged menu and a phone number to call to make reservations. For a conference, you might have the schedule for that conference, the venue and the name of the organizer in case someone needs to get in touch with someone and just can't get the web page to pull up or they're having connection issues, which is pretty common in hotels and conference venues where a lot of people are using the Wi-Fi and things kind of break or go down. Extending this a little bit further, you can cache pages in real time as people visit them, and then if for some reason they go offline, you can show them a list of the pages they visited and make those available to them even though the site is offline. And this is particularly useful for reference sites, news sites, social networks, utility apps. Just because the site is offline doesn't mean people can't still use it and access things that they've already been to. You can use service workers to cache core assets for performance reasons. Things like CSS, JavaScript, images, fonts, any of these really heavy, commonly used files, you can save them locally and then serve them from the network, from the cache rather, instead of going out to the network. This is also really great for people who are on low data plans or live in area where downloading things takes a really long time. This can dramatically speed up performance. Once you have that asset saved, you can serve it from your cache over and over again, and it returns instantaneously instead of taking a few hundred milliseconds or several seconds depending on what a person's connection speeds are. And as you may have picked up by now, you can use these different techniques in conjunction. So you don't have to only choose offline first or network first. You can use those different approaches for different types of assets. So for things like CSS and JavaScript, you may go offline first for those, and then you may use network first for your HTML and API requests and have those as a fallback when things go offline. Now, on the more extreme end of things, you can go fully offline. And this is great for things like apps and games, where once you download the initial assets, the HTML, the CSS, and the JavaScript, you have everything you need. You never need to go back out to the network. So think about a utility app like a calculator or a game like a JavaScript-based Pac-Man game. And if you pair this with a manifest.json file, you can turn the simple web app with a service worker into a progressive web app that users can install onto their home screen and then load fully offline without any of the browser Chrome so it functions like a native app, but it's a web app that works completely offline. Now, my personal favorite use for service workers is replacing single-page apps, which tend to be really fragile and easily broken, with multi-page apps that are more resilient and, in my experience, actually provide a better developer experience. With a single-page app, the whole site or app exists in a single HTML file. JavaScript renders the content, handles URL routing, and so on. Now, the thinking behind this is that because only the content refreshes and you don't have to re-download all of the JavaScript and CSS, each page loads faster. And if you have an API-driven site where you're getting data from an API request and then using that to render your content, you can hold that in memory as the pages change and you don't have to constantly make new API calls every time the page reloads. But as we've already learned, service workers can give you a lot of those same benefits. The problem with single-page apps is that they break a bunch of stuff that the browser just gives you for free out of the box, and you need to go recreate it with JavaScript. For example, you need to intercept clicks on links and suppress them. You also want to detect if the user right-clicked on the link or command-clicked and they're trying to open it in a new tab or window and allow them to do that. A lot of single-page apps break this experience and break user expectations. Once you detect that click, you need to figure out which HTML to show based on the URL path. Then you need to update the URL in the address bar without triggering a page reload because that would defeat the entire purpose. You need to handle forward and backward button clicks, which again a lot of single-page apps just kind of forget about. Update the document title, shift focus or scroll position on the document depending on where someone's supposed to be. Again, a thing a lot of single-page apps break. Then you really want to shift focus back to either the document or more ideally the heading element so that screen reader users get an announcement telling them that the page has changed and they know where they are. This is another thing that a lot of single-page apps forget about and break and make the experience bad for their users rather than better. One of the things people love about single-page apps is that if the internet connection drops, you can still use the app and page loads feel instantaneous. But when paired with service workers, multi-page apps give you those same benefits but with more resilience and less developer complexity than a single-page app. You can still render HTML with JavaScript. This approach is aided by static site generators, which let you combine templates and markdown files to pre-render hundreds or thousands of HTML files. Because they require no server rendering, they load fast and they can be cached easily with service workers. When you pair them with CDNs, you make the experience even faster. Now, when I talk about this, I always get asked about the API piece of it. You're making a call to an API and with a single-page app, that would just live in the browser memory and you never have to worry about it again. But with a service worker, you still make a call out to the network just like you would the first time you load a single-page app. And then you cache that JSON response in your service worker's cache instead of just trying to hold it in memory for the entire session. The caveat here is that you usually want to put an expiration date with it. So you want this to eventually go away, either when the user logs out or when the session ends or after a certain period of time. And then every subsequent request, every page load for that API, you load it from the service worker cache instead of reaching out to the network. And this effectively gives you that same instantaneous response that you would get if you were just holding it in browser memory, but without all of that developer overhead. Everyone who buys one of my courses or workshops or e-books gets access to a portal where they can download all of their stuff. And it's a multi-page app built this way. The logo, navigation menu, and page heading are all baked, hard-coded into the HTML using a static site generator. The content varies from user to user, and that comes from an API call, which is loaded dynamically with JavaScript. I've recorded a video of me navigating through this portal in real time so you can see how fast the experience is. Most of my students actually think it's a single-page app, but it's not. It's a multi-page app, and the pages load seemingly instantly because they're using statically rendered HTML and API data that's being served locally from the cache. So it's instantly available as soon as it's requested. And it results in this really fast experience that was easier for me to build, easier for me to maintain, and is way more resilient than the single-page app version of this would be. And I know because I built both versions and compared them to see which one worked better, both from an ergonomic experience for me and a resilience perspective for my users. So if you remember only one thing from this talk, what I hope you take away is that by using Service Workers instead of the fragile house of cards that we have today, we can build a faster, more resilient web that works better for everyone. If you found this talk interesting, I put together a bunch of resources for you on Service Workers over at gomakethings.com.jsnation. You can find the slides from this talk as well as a ton of related articles, podcasts, books, and more. Thank you so much. It was really great chatting with you. Whoop, whoop, whoop, whoop. That was such a good talk. I want you to go over into the community channel and throw us your best emojis to congratulate Chris on such an amazing talk. Really, really loved hearing it. And also, one thing we're going to need to do before we move on is we need to go and find out the answer to the question that Chris posed. Chris asked us, tabs or spaces? And I'm just going to get the poll up right now and check what the answer was. So, tabs or spaces? And we can see that a majority of you wonderful people have said, let's see, tabs, which is brilliant. Thank you. You're like-minded. You're just like me. I'm legitimately, I'm both pleased and surprised by this. I'm bringing in Chris now. So you're pleased and surprised. So what do you choose usually yourself? I'm tabs. I'm tabs all the way. Chris, I like you already. I like you already. It's just objectively the best. Absolutely. What are people with spaces doing? It's efficiency. I'm usually the outlier here. Pressing it once versus pressing it four times. Come on. I'm joking. I am very passionate about my love for tabs. They're all developers here. But Chris, thank you so much for that talk. We really appreciate it. And we've got lots of questions that are coming in. So make sure also if you're listening and you still have a question that you want to ask, drop it into the chat, into the Q&A channel, sorry, right now. And we will make sure we get all of those asked. So I'm going to go straight from a question that we have from Alexius, who asks, can I intercept or get requests for a particular API endpoint and add or modify a search parameter of this request? Sort of, yes. So and I reserve right to be completely wrong here because I haven't tried this myself. So browsers may throw up some sort of like security thing around this that I'm not aware of. But the limitation here with API requests is with service workers, you can only intercept get requests, not posts, puts, deletes, anything like that. But using this approach, you could theoretically get the request and then use the fetch method in your service worker to call that same endpoint with a changed parameter or an additional parameter or, you know, some sort of like additional variables put on. And then when you get the result back, pass that along. You don't even have to cache the result. This could just be a thing that you do in real time as a way to, as the title of my talk suggested, run a man in the middle attack on your own site. But, yeah, conceptually you can absolutely do that. I'm not aware of whether or not browsers might, you know, raise their hackles a little bit about the idea of that kind of thing for security reasons. But I believe that that is absolutely something you can do. That's awesome. That's awesome. I love when you talk about it as a man in the middle attack on yourself. I don't know why. It just makes me laugh. We've got another question. And this one I actually want to know the answer to as well. CRS1138 says that this is a cool talk, but have you got a tutorial on ServiceWorks? You know so much and you're really good at explaining it. Right. Great question. Let me find out. That sounds like a really silly thing to say. I write so much that, yes, I do. I have a handful of them. I will make sure, it looks like I have nine. So I will drop them in the Discord when this Q&A is done. And I'll make sure to reply directly to your message so that you see it. Yeah, because I have a whole bunch of them. I also have some recommended books and other things like that that you can also check out. Specifically Going Offline by Jeremy Keith is where I learned a majority of what I know about service workers. And it's an amazing book that I highly recommend. I'm going to check out that book. Thank you for the recommendation. We got another question from Phil Don saying, can service workers be gradually adopted on an existing code base, perhaps on a one by one endpoint basis? Absolutely. And on an existing code base, I actually think adopting all things gradually is probably the best way to go. It's a really nice kind of thing where you can start by layering in just some base functionality and then you can make it progressively more complex as time allows. So, for example, I think one of the easiest kind of low hanging fruit things you can do with service workers, if you don't have any on your site at all, is to use them to cache long lasting, frequently accessed assets. So CSS files, JavaScript, images, things like that, just to kind of give your site a nice little performance boost. And then from there, you can start to layer in more stuff like caching endpoints or providing an offline first experience or, you know, even like kind of a fallback experience so people can access things they've previously viewed. But you can kind of throw those in later after you've gotten that base experience in place. That makes sense. That makes sense. And someone's asked, speaking of caching, they say cache is the best thing for most use cases. Yes. But can you please help them understand how and when they should invalidate things? If there is a new version of the site, how effective would it be? And is it easy to implement? Yes. Yeah. Great question. I didn't really touch on this all in the talk at all. Time limitations and all that. So service workers are super magical in that every time the browser loads one, the service worker itself is also cached. But whenever it loads, if there's an Internet connection and the browser can grab the newest version of the file, the file name doesn't even have to have changed. It just will find the current version of the file and compare it to the existing one. And if a single byte of the new file is different from the one that it has cached, it will, in the background, replace the old file with the new one and automatically update that for the user. There are also some. So the caveat here is it won't actually use that new file until the visitor has completely closed out any open tabs with your site or app in it. There are some tricks you can use to force it to start using the new file sooner before that install process happens. The code for that is actually in one of the articles as part of a series that I wrote. So I'll make sure that I drop a link to that in the residence track Q&A so you can access it. Just because the URLs get kind of too long for me to read out loud here. You know, you're really good at educating people on the Internet whenever you answer a question with I have an article for that. That is so awesome. We've got another question from Kev that are saying that this is a great talk. I'm a web developer and I guess you can also use a service worker to mock an API get request. I'm pretty sure you can do that, right? That's a great question. Yeah. So theoretically, if someone kind of calls an endpoint that may or may not actually exist, you can intercept it and then respond with like you don't actually have to make a live call anywhere. You can just create a new response using the new response object and send back whatever data you want. Yeah, absolutely. That's actually kind of a neat thing I hadn't really considered before. Yeah, that sounds like an interesting way to use it. And Tom Rafa also asked, are there any situations where you would advise against service workers, apart from fetching frequently to update data? That is a good question. Oh, where I would recommend against service workers. No service workers are for everything like service workers are one of the best things to ever happen to the Internet for a variety of reasons. I think there are different use cases for service workers, and so the way you implement it in a particular situation might be different from another. So caching aggressively might not always be appropriate for all the things. You know, for example, but I can't think of a single use case where a site or app wouldn't benefit from having some form of service worker, even if it's just doing a little bit of stuff. There's a very extreme end where you're caching everything and just going straight to the cache first. And then there's like the more lightweight experience where you provide some sensible fallbacks. And then there's a whole range of stuff in the middle. And which strategy you choose is going to depend heavily on what your app does and how much time your team has to commit to kind of building out this approach. Yeah, I also get that where you've got a balance of what's like sort of the ideal sort of thing that you can create and then actually how much time and resources you have to get there. And when you spoke earlier of the gradual approach, being able to sort of use your resources to slowly bring that in, it makes so much sense. And then someone asked the question, which I think is quite interesting. Why is it that not many apps go service workers and offline first in the real world? What do you think it's stopping a lot of other websites from sort of implementing this approach? I think a couple of things. First of all, they're not really new. Like they're newer than a lot of other things that we have. But I don't feel like they're as well known as some of the other kind of technologies and solutions we have. And I also feel like for a really long time, single page apps have been such a thing that that's just the tool people grab for a lot of things. And so a lot of what we do on the front end, not just not using service workers, but just a lot of the choices we make, often feel like a byproduct of developer inertia, rather than necessarily being the best tool for the thing we're trying to accomplish. That makes sense. I mean, I'm not going to lie. I remember when one page apps came about, I loved them. And this is kind of a question that I have for you as well. Because you spoke about how that sort of single page apps can be bad for a couple of different reasons. But what would you say are the times when that's actually the right way to go? Yeah, so I think one example I can point to where single page apps maybe make sense is if you have some sort of like real time chat app. Twitter actually seems like a really good example here where you've got a lot of different things coming in in high volume. Doing all of that asynchronously in the browser, just kind of fetching the new stuff, showing it, creating links to the tweet with JavaScript. That's actually probably more performant and more scalable and more sustainable than trying to do that with hard coded HTML on the server and caching everything with service workers. I think you can still use service workers with something like that to provide some really good performance benefits and some offline fallback. So anybody who's ever loaded a native app, and all the tweets you were looking at just before you last quit are automatically loaded there. That's a great thing you can do in a web app with a service worker. But I actually think something like a Facebook or a Twitter probably functions better with, or even like the web version of Slack or Discord, for example. They probably function better as single page apps than server rendered apps. Although we're starting to see some interesting tools that kind of blend the borders there. It's so interesting sort of watching as people use the internet in different ways. At first we kind of all think that there's this one right way to do it, and then we're beginning to realize that different approaches really fit different use cases. We've got so many more questions. People love this talk. Someone asked, would you usually notify the user if a new version is deployed, or would you just wait for the service worker to be reinstalled in the next session? So I say this as someone who uses service workers in a very kind of basic and light way. I've not really found a need to do that. Usually the changes I make to my service workers are not substantive enough that they warrant like you need to reload the browser right now to like make this thing kick in. Or I can force it to kick in kind of behind the scenes. Usually what's happening with my service workers is they're just kind of caching assets as they come through. And I may make an update that caches some additional stuff or stops caching some stuff I was caching before. But they're not doing the kind of like this will make or break an application type of work that requires me to like show a notification for a user. There is a way to do that. Using the post message method, you can actually send events to service workers and then from service workers back into the client. And in both files, you can listen for these events and kind of react to them accordingly, which is pretty neat. So if you did want to do something like that, where you like show a message in the browser, that's absolutely something you can do. I'd love to check that out. Actually, I've never I've never actually thought about sort of that, that using it in that way. Now I'm going to go and Google it and find an article for me about it. But thank you so much, Chris. It's actually been a pleasure to talk to you and to learn because you're so knowledgeable about service workers. But since you spoke about the tutorials you write, where can we find these tutorials and where can we find out more about the things you write about? Yeah, so I would recommend if you would enjoy this and you want to learn more, if you head over to go make things dot com slash JS nation. I have put together a ton of resources on service workers, recommended articles, books, forms to sign up for my newsletter, as well as the slides and video from this talk. So you can find that all on on my website. Awesome. Thank you. And me is so nice. We have perks from companies and sponsors, but we even have perks coming from speakers, too. So definitely check out his website. And I'm going to check out right after this after this event. I'm super excited. But thank you so much, Chris, for hanging out. I hope you've enjoyed it. I have. It's been a blast. I'm going to head over to spatial and if anybody wants to chat. Cool. Yeah, definitely. Join Chris in the spatial check. The link will be in the timeline below. See you there, Chris. Cheers.