Mock Service Worker 2.0
AI Generated Video Summary
1. Introduction to API Mocking with MSW
First things first. My name is Artem. You can find me as Ketanaito pretty much everywhere, including my humble blog at ketanaito.com. And you may also know me as the author of MockServiceWorker.
Out of curiosity, can you raise your hand if you've heard about MSW? Wow, that's a lot. You will enjoy this talk. Yes, so for those who haven't heard about it, MSW is an API mocking library. So in essence, it allows you to intercept requests and mock responses, basically control your network for whichever it is you want, testing, prototyping, debugging.
But how do you do that? How do you describe the network with MSW? Well, you use these functions called request handlers. I will show you an example. So this is a very simple request handler. You can see that the intention here is to intercept a POST request to the slash user endpoint. And then we have this callback function that should produce the response. We have request, argument, response, and context. So what we're going to do with this request, we will try to read its body as JSON. So MSW will try to be smart here, actually, and see what content type header your request has.
And if it's application slash JSON, it will try to parse it and give you the object back. So we're accessing the name property here. And then to return a mock response, we use this REST composition function to compose a bunch of context utilities, like this context.json to return a JSON response. So this is a very, very basic example of your network description.
3. Motivation to Improve MSW
4. Challenges with Isomorphic Requests and Responses
As much as amazing it is, it can be better. You should mock your network in the same way that you use it in production. In the browser, you can use Fetch API or XMLHttpRequest. In Node.js, there are more ways to make requests than there are stars in the sky. MSW tries to bring all these different ways to the common denominator with isomorphic requests and responses. However, it turned out to be quite a mess, as developers want to handle various features when working with the network.
As much as amazing it is, it can be better. So the primary perspective there was one of the key points that we had in the library from the very beginning. You should mock your network in the same way that you use it in production, for example. So let's take a look at how you do that.
In the browser, most likely using Fetch API or an abstraction or a Fetch API, it's a fantastic API. Or you may be using XMLHttpRequest, which is still great for certain situations, albeit being a little bit older. And in Node.js, it's a bit more tangled. The main way to make requests in Node.js is the HTTP module, which you're probably not using directly for a good reason. So you rely usually on third-party libraries to do requests, to handle responses. And one particular thing I learned when writing a request interception for Node.js is that there are more ways to make requests in Node than there are stars in the sky. And unfortunately, that's true, especially when you dive into like user land and what people are building their own libraries and they use Node.js in all sorts of ways, sometimes according to the specs, sometimes not. But what we try to do in MSW is to take all these different ways you can define requests and responses and kind of bring them to the common denominator. And we introduced this concept of isomorphic requests and isomorphic response. Basically, again, an abstraction just to handle all these differences. But it turned out to be quite a mess. Because yeah, you can represent very basic requests and responses, but there are a lot of features that developers want when working with network. You want to be able to handle array buffers, files, form data, uploads, streaming, maybe abort the request. And we had to kind of implement it by ourselves. So no matter how you declare requests, you still get those capabilities. It was a huge mess.
5. Node.js 18 Release and Refactoring
Node.js 18 was released, introducing the fetch API as a common ground for representing the network in both browser and Node.js. This allowed for refactoring and simplification, removing unnecessary code and documentation.
And just when I was losing faith in all humanity, something amazing happened. Node.js 18 was released. Now, this was a grand release, especially for me, because I was deep down in refactoring in MSW and we still promised support for Node.js 14 and 16. So there was no common ground really to represent network until this happened. And of course, I'm talking about one of the biggest features in this release. It was the fetch API, this very fetch function that you are all familiar with, I'm pretty sure. But it's not only that function. There's been a lot of other things added, like readable streams, headers, working with abort signal and URL. And if those things were not introduced by this release directly, they were facilitated. They were much easier to use through the fetch API. So this was amazing. I was very happy and I thought, hey, we can use fetch API as the common ground for both browser and Node.js to represent the network. And I began refactoring and employing one of my favorite practices, throwing code away. And it's been fantastic, honestly, just throwing all the subtractions away and all the docs because you don't need them anymore.
And then a year later, we arrived at MSW 2.0. On the surface, it's pretty much the same request handler. This is, by the way, the same request handler, just with a new API. So we still have the intention to intercept the post request to the user end point. But in the callback now, we just get this request object. And you may be wondering, like, what kind of abstraction is that?
7. Benefits of MSW and Support
You can apply the knowledge gained from using MSW to other areas of development. Open source maintainers make mistakes and constantly improve. Try MSW by installing it with npm. Dive deeper with the Mock REST and GraphQL APIs course. Support the project financially or through contributions.
And of course, you just get better at the language. As I mentioned before. So, you interact with this primitives more often. You get more familiar. And this knowledge that you take from using MSW, you can apply it even outside when I don't know, writing loaders in Remix or handler requests in Next. Those all rely on the same Fetch API.
And despite common belief, open source maintainers aren't this wise, all-knowing creatures. We do make mistakes and we do have to iterate and improve a lot. And the more we ourselves interact with the language, the better APIs we can design for everybody to use. If you are excited about this direction, or if you've never tried MSW before, you can do it right now. Just npm install MSW and you can mock any APIs anywhere across your entire stack.
If you want to dive deeper, though, I spent the last year working with Ekhed on this fresh new course that's called Mock REST and GraphQL APIs with a mock service worker, where we're going to build together a movie streaming app completely mock first, so against mock servers. And we're going to dive into the basics of getting started with the library and intercepting your first request, to more advanced concepts like mocking streams and dealing with GraphQL queries and mutations and even schema-first approach in mocking. So it's a fantastic place to get started. I encourage you to watch this course and I'm sure you will learn a lot.
Of course, you can visit the project itself at mswgs.io and on GitHub, too. Read about it and if you believe in the mission we're doing here, support it both financially and contribution-wise, which is really the best support. And yeah. Thank you.
Missing Features and Request Mocking
There are some missing features in MSW compared to older alternatives, such as an expectation layer for requests. One anticipated feature is VAPSocket API support. Auto-generating MSW code by monitoring network requests is possible through ecosystem packages, but native support may use HTTP Archive. MSW does not mock requests but uses the ServiceWorker API to intercept and respond to requests.
So, we'll get right into it. There's quite a few, like I said. The first one here. So, are there any big features that are currently missing from MSW when compared to older and maybe more mature alternatives? I think there's a lot of expectations features, and that's a pun. Like people are used to, for example, using NOC in tests, and sometimes people want to have this expectation layer. Like you want to perform a request, but also expect that it has a particular payload or it has been performed once. This is a deliberate choice that MSW doesn't ship this and instead I would rather have an ecosystem package that extends the usage, because not everybody needs these expectations. And so, this is something that's missing. And I think one of the most anticipated features that I hope I will have time to work next year would be, of course, the VAPSocket API support.
Okay. Great. That's good to hear. For sure. This next one here. Can we auto-generate MSW code by monitoring network requests from our application and updating them when the response structure changes? This is a really good question. I know a lot of folks are doing like crazy magic, like deriving MSW code from your GraphQL type definitions, from your open API specifications, from your packed contracts. So, you can find all of that online. There's again, ecosystem packages, people building this. As of native support, I think, like letting you on a little teaser, I would much rather have HTTP Archive as the common ground. So, maybe in the future, there would be something around that.
Okay. All right. We'll keep an eye out for that, for sure. No promises. Okay, so this next question. Why not just mock the response instead of mocking the whole request? Hmm. I'm not sure I quite understand the question, but I can elaborate a little bit. So, with MSW, you are not actually mocking requests. This is something I briefly touched upon on one of the slides that we use the ServiceWorker API, if talking about the browser interception, that allows your application to actually make requests. So, there's no stubs, and your requests happen and arrive at the network and just receive response from the ServiceWorker.
Support for TypeScript and Contract Testing
The only thing you're mocking is the response. MSW always supported TypeScript, but more materials are needed. MSW is written in TypeScript natively. MSW can be used with advanced tools like PACT I-O for contract testing on CI pipelines. MSW is designed to work well with other tools and achieve various use cases. The compatibility with the standard API is a focus. Without Node's changes, the approach would have been different, possibly contributing to Node with FetchAPI.
So, the only thing you're mocking is the response. The request part is there just to have this contract of request-response collocation. Yeah, okay. That's good to clarify then. Yeah. All right.
Does MSW give support to TypeScript yet? MSW always supported TypeScript, but I think my fault was not providing more materials on that. I wrote one article on Dev.Too, which I guess is one of my most viewed articles, and I should have known better, about using MSW with TypeScript, but it was, like, based on the previous version. So, in the new version, I do want to do something better about it. It would be nice to have code examples in the docs that feature TypeScript as well. So, this is a contribution tip for you, if you want to add it. Yeah. So, it will definitely work better, but MSW is written in TypeScript natively, so it always had this generic support for request body, response body, body parameters, and even GraphQL queries and variables. All right.
So, if you're new here, docs contributions welcome. Always a great place to get started with open source, if you haven't yet. So, yeah. The questions just keep coming in, so we'll keep going through them. Can you provide a use case with MSW and contract testing on CI pipelines? Yeah, absolutely. I think MSW is just a way to facilitate contract testing. So, you may be, like, in itself, it doesn't do that because it's a very simple tool of request response contracts, but you can use some more advanced tools like PACT I-O, and you can find a way to integrate them. I know that the PACT team has an official integration. I believe it's PACT-MSW that you can find on npm. And I believe it utilizes MSW as just an interception algorithm and uses PACT for contract testing and CI. So, you absolutely can do that, especially if you like contract-driven testing. Go ahead.
So, it sounds to me like, really, MSW is designed to do, like, what it does very, very well and work really cleanly with other tools to achieve whatever use case that you need. Yeah, and one of the ways that we try to improve that is exactly betting on the same standard API so everybody gets the compatibility. Yeah, absolutely. As I'm curious, if Node hadn't come out with these changes, do you think that you would have taken a different approach at some point or just kind of kept working on trying to make it as usable and compatible as possible? I think I would spend another two months pulling my hair out, and then I would contribute to Node with FetchAPI.
MSW Usage in Testing and TypeScript Benefits
Using MSW in different testing levels depends on the product and its requirements. Conventionally, network code is tested in integration tests, while units should be isolated from HTTP interactions. MSW can be used heavily in integration tests and even in end-to-end tests. TypeScript is crucial for maintainers, providing static tests and ensuring the functionality works. It contributes to the developer experience and has no runtime features. Teams can also explore GS doc for type comments.
Okay, there you go. Yeah. You know, if it's not getting done, do it yourself, I guess. But luckily, like you said, it got done, so that's great to see.
Okay, this next question, do you recommend using MSW with unit tests or only with integration? I guess just as with testing levels, it depends where you draw them and what kind of product you're building. Conventionally, you test network code in your integration tests. Your units should really be isolated, and that includes isolation from HTTP interactions. So people do test heavily and use MSW heavily in integration tests. You can also use MSW in end-to-end tests. I'm particularly fond of this concept where you run your end-to-end test suite against mock and then against production. It can help you catch the little deviations. But of course, if what you're building really has to have network code in its units, I suppose of course it's up to you. Yeah, I think we're required to have one it depends answer per Q&A. So that's a... There you go. Because it always does. It always depends, right? Yeah. Yeah.
All right, great. Thanks for sharing that. So this is a good one here. What's at the top of your wish list, so maybe if you were to build it yourself or wish for it, for the future of JS API request response fetch features?
Oh. Hmm. I do wish we got upload progress to fetch API. That's probably one thing that makes me use XMLHttpRequest still. And I hope that maybe one day we will. Okay. And hopefully too. If not, maybe we'll see a PR.
All right. And what do you think about Chrome developer tools and mocking?
Yeah. I actually heard that the team at Chrome introduced a lot of cool things recently. You can do mocks straight in the network tab, and then maybe even save them or something. I think it's really cool. But I see this as primarily a debugging instrument because it's faster. It's more accessible. It's DevTools in the end, so it's your tooling to debug and build. But as far as the comprehensive solution, I still believe that API mocking doesn't belong to a particular tool, even if it's a browser tool. It can be a cool feature, but if you are building the whole product, you are testing the whole product and you want this network description to be consistent, just like I demonstrated on the slides. It's not just plugs in different areas. It's just one layer, then reused everywhere else.
Yeah. And it sounds like you want to have that kind of intentionality from the beginning when you're designing that out, right?
Yeah. Yeah. So kind of more proactive versus reactive, so to speak.
Support for WebRTC and Documentation
There's a lot of cool stuff in Chrome DevTools. MSW provides primitives for users to build on. It's important to draw the line between library responsibility and user requirements. Instead of building something that does it for itself, illustrating use cases with documentation and examples is a better approach. The rebuilt MSW website has new features like search and more recipes. Schema-first mocking is possible with mock resolvers, schemas, and types.
Yeah. That makes a lot of sense. But yeah, there's a lot of cool stuff in Chrome DevTools these days. For sure. Yeah.
All right. So will you support mocking WebRTC? I think there is an issue about it for like dating two years ago or something. Maybe it's something else. I think the stance on MSW, whatever support you bring to it, is very simple. It should provide you the primitives and you can build on top of those primitives to achieve whatever you want. So if you want DRPC support, I'm almost certain you can do this with MSW right now. But it doesn't mean that MSW should ship it natively. It's quite an important decision to draw this line where the library responsibility ends and where user land begins with all this crazy requirements and use cases. Yeah.
Like we were talking about earlier, right? It does what it does well. Yeah. It integrates with other tools. And do you think that there's... Instead of maybe trying to build something that does it for itself, maybe just illustrating those use cases, like with documentation or examples?
Absolutely. Like one thing that we can do is just show examples and we actually rebuilt the website over the past year. So it's a completely new website. It has docs. It has search. Oh, that's a big one. Yeah. And I'm trying to add more and more recipes, like streaming and like polling, using generators, a lot of stuff that I know people are using, but not everybody knows about. Like I know that we had a chat with folks from Apollo and we were iterating on how to improve schema-first mocking. And over the chat I realized that I'm not doing a good enough job to promote that you can do schema-first mocking even right now. You can do mock resolvers, mock schemas, mock types. It's just a documentation issue.
Different Approaches to Mocking and Testing
There are two types of developers or testers: those who only use documented features and those who use everything, even inappropriately. As a maintainer, my role is to bridge this gap and provide guidance. Mocking is not inherently evil, but a technique to establish boundaries. MSW addresses the historical perception of hackiness by using service workers and mutation modules. It depends on what you're testing and how you built it. Like any tool, it's about how you use it. Now, let's move on to the next question.
Yeah. I feel like there's kind of two types of developers or testers. Some will... If it's not in the docs, they don't think it exists or they're afraid to use it. And then some other people who will just use it for anything and everything, including maybe things you shouldn't use it for. And so, yeah. So I think that there's some people who probably have already figured out that it works together, and there are other people who could use a little bit more guidance.
Yeah. I think as a maintainer, my job is to bridge this gap and bring these people closer. Yeah, absolutely. All right. We still have more questions and have a little bit more time for them. So this next question. So Venkat Subramanian said once, knockout before you mock out. I haven't heard that one. Could we fix the root problem in Fe with better architecture and dependency injection? I think there's nothing particularly wrong with mocking. It's just a technique to establish boundaries for you to tell the test where the test should end, where your system should end. So on its own, it's not evil. I know it historically has this air of being hacky and stabby. And this is something we are trying to solve in MSW by using service workers, by using a module like mutation in OGS. But I guess it's still another one. It depends on what you're testing and how you built it. If you constructed your app to be friendly to dependency injection, then of course you can utilize it during testing. But you need to acknowledge this creates a different testing boundary now.
Yeah. I think it's like you said, with any tool or approach, it's not inherently good or bad in its own, but how you use it. If you have a hammer, it can be used really well for some things. Maybe don't do it for something that requires more precision and that it doesn't make a hammer good or bad. Exactly. And so, yeah, I think, again, we'll give you two this time for this Q&A.
API Automation and Integration with Postman
The best tool for API automation depends on the use case. Postman is popular, but there are many options available. AI can enhance testing, but it should be used responsibly. Controlling test data can be challenging, and automating the process would be beneficial. Integration with Postman is possible, although the specifics may vary.
It depends. So, all right, this next question, what is the best tool for API automation according to you? API automation. I wonder if you mean testing or like complete workflow? I know that Postman has been around a lot. I use it in the past a little bit. I don't really know what to recommend. There's a lot of them out there. I think I'm going to say it. It depends, right? It depends on what you're looking to automate, what the use case is that you are trying to automate for your API. If you're doing automated API testing, if you're doing any kind of like migrations, I think it will just depend on- Yeah, especially like with the rise of AI right now, I'm pretty sure there are some startups building really cool, like semi-intelligent features of like scrapping your API and letting you discover them. Making sure that new projects are rising every day. Yeah, absolutely. And since you brought up AI and there wasn't any specific questions about it, how do you see that kind of maybe potentially impacting how we think about day test data or mocking or in any kind of way? Is that something that you would see integrating well or not? I think AI just as mocking is just another tool you can use to supercharge your experience. I use GitHub Copilot or CodePilot when testing. It does help writing repetitive tests when it's kind of easy to predict what I want to test, but for more complex tests, I would rather turn it off. So again, it's a tool you need to know. It's your responsibility when and how to use it and to understand that you sometimes need extra to pay extra attention to what the AI generates here. Yeah. And I think, I know I mentioned this before, but one of the reasons why people go to mocks is because obviously controlling your test data can be very difficult and generating the test data and creating the seeds. So if there's any way to do that from either test code or application code and handling all that for you, that's something I think would be a very good problem to solve because it's a painful, painful problem. It can be. It can be, yeah. All right. I think we have time for one more question. If you have any other questions for Artem, make sure to get them submitted in Slido. So is an alternative integration with Postman possible? I'm not sure. I suppose it implies MSW and Postman. I think so, yeah. I frankly haven't used it in a very long time to know where can we draw this common line. I'm sure it's possible again, because MSW is a simple tool. And then Postman is a much more comprehensive tool.
Postman Support and Conclusion
I hope Postman supports network archives. Give it a try. Postman is available for questions and chats. Feel free to engage with Artem for Q&A. Thank you for the presentation!
I hope Postman supports something like network archives. And this is again like where we can draw the common line maybe. So I don't know. Why don't you give it a try?
Yeah. And Postman's here. So if you want to ask them any questions or chat with them, let us know. Well, if you have any other questions or if I didn't ask your question the right way or we didn't address it, please feel free to chat with Artem, speak your Q&A, and hear around the conference.
But yeah, thank you so much for your great presentation and for being here. Yeah. My pleasure.