Transcription
♪♪ ♪♪ Hello, everybody. Thank you for joining. I hope you are healthy and well and would love to hear about API mocking. My name is Artem, and I am a JavaScript engineer, which has a huge love for open source. I've been doing open source for more than five years, during the course of which I was extremely lucky to author more than 20 packages that by this time have more than 8 million downloads on NPM. And today I would love to talk with you about testing and specifically about API mocking and the role it plays in it. But before we begin, I'd love to ask you a question. Why do we write tests? Well, we certainly don't write them to gather code coverage. It can be useful at times, but unfortunately, it doesn't mean that our product works or works well. We're not writing tests to check each and every little piece of logic we wrote because that would be testing our code instead of testing the intention behind the code, which we shouldn't do. I think why we write tests is first of all to gain confidence, to rest assured that the software we build is functional and our customers are enjoying and loving it. And to gain that confidence from testing, we should follow two principles. And the first one is to test as a user. And the user here doesn't mean customer necessarily. Of course, if you're having an e-commerce website and you want to double check the success scenarios in different pages, you will be performing user actions there. But let's say you're writing a test for a function or a class. Then the user for that class would be another developer. So you need to put their expectations against your class and write your test following by that expectation. I like this quote by Ken C. Dodds that says, the more your test resembles the way your software is used, the more confidence they can give you. And I think it summarizes this principle pretty brilliantly. The other one principle that can gain us more confidence during tests is to establish clear boundaries. I think you're pretty familiar with this because this is the reason why we have different testing levels. So when we want to focus on a single unit of code, we write unit tests. Then perhaps we want to check that pieces fit together nicely and we do some integration testing and we can wrap up with end-to-end tests to check the entire system. And I see mocking as a tool that distributes these boundaries. Let me give you an example. Let's say we have this orange square and we want to test it. Now, in real application, this orange square probably does quite a few things and it may depend on other squares like this blue one. Now, because of this dependency, we can no longer focus our tests on the orange cube alone and we need to somehow account for the blue one. So this is where we can mock the blue one to substitute it with a seemingly compatible cube, but it's going to be different. And because of this, we can control this dependency and connection between modules and make sure that our test of the orange one is focused. And API mocking is a technique that allows us to substitute API communication, so HTTP requests, in the same manner. There are two main practices when it comes to implementing API mocking in your projects. The first one is to use a mocking server. This is a pretty straightforward setup and it means that you have a standalone HTTP server that will substitute a production one. And while it's fairly easy to set up and it has some sort of abstraction syntax to write the routes and responses, I think it has a few disadvantages. So primarily, no matter what are you going to use, write your own server or use a third-party library, you're going to end up pulling dependencies and the whole process will feel like you're writing an actual server, but you're not doing so. Then you need to ensure the operability of your server, so it starts and stops at the right time before or after your tests, and then there are no runtime exceptions that can crash the tests. And the worst thing of mocked server is conditional URLs. And this is what I mean by them. This is an abstract example of code where we have some fetch call to an API backend.com. Now, we don't want to hit that production URL during testing, so we probably introduce some environmental variable that says that, hey, if you're running under test, just hit this other URL at localhost, because this is where we have the mocked server running. Now, the issue with this is that during the test run, the code will never hit this line, which it does in production. And let's say we made a mistake here. We mistyped a protocol or we missed a few slashes or dots. Now, this is going to result into a perfectly passing test on CI while perfectly crashed app for our customers. And the reason that happened is because we introduced a deviation. So the app that run during test is slightly different than the one that runs on customer machines. And the more you alter the system under test, the more you're testing a different system entirely. I think the other approach how to do API mocking is to use a mock client. So this is a quick example of spying on window.fetch and saying that each call of fetch should resolve into this JSON, which we specified. And what this is going to do is that instead of calling the actual fetch function in the browser or during the test, it's going to always call this mock and return this fixed JSON. Now, probably we don't want all requests to return the same JSON. So we want to introduce some logic where responses depend on requests. And we end up writing this API client mock, which has this logic. And then we realize that we also need to account for some business behavior and also keep account for headers and other things to make this API client mock feel realistic. And in the end, we end up with a huge obstruction, which we wrote just for the sake of testing. There are also a couple more issues with mocked clients. For example, because we're taking out the client and replacing it, we need to ensure that requests and responses make sense, that they are spec compliant, and they are similar or identical to what is going to happen in the actual app. The input to such mock client is also always considered valid. So unless we somehow validate the request and responses, the mock client is going to digest them and perform them, even if they have a typo, for example. Then clients are quite different. And the way response and requests are handled in fetch may be different for XHR and Apollo. So we need to take account on that as well when writing it. And this creates a very tight coupling between our mocking and actual libraries we use, which makes it very hard to migrate and to see our software evolve. The worst thing is that requests never happen with the mocking client. That's kind of what we wrote, right? So instead of calling actual fetch or Apollo, we're calling the mocked counterpart. So the actual logic never executes. Let's keep that in mind and think about what could be a better approach. So here we have an app. It can be an actual app running in the browser or just a part of an app running an integration test. The point is that this app makes a request. Now, we want this request to happen. We want the actual request client to be called and request to leave the app. Why? Because this is what happens in production. And if we stay close to this, we will get more confidence from a test. But we don't want for this request to hit an actual server. That way, if the server is down or malfunctioning, our test will fail. The point of our client-side test is to test the client, right, not the server. But we still want to receive a mocked response back so we can control different service scenarios per test and it will be reproduced reliably. So we always tend to choose either mock client or server. But there's actually a thing in between so we could choose neither. And that thing is called service worker. Service worker is an API that is a kind of a web worker, a JavaScript module that lives next to your application in a browser and executes in a separate thread. This is an example of a worker file. So workers have certain lifecycle events, and one of them is a fetch event. Worker triggers this whenever your app makes any kind of request. And in this handler, I'm writing that I want to look up the cached response, and if there is a cached response, just respond with it. So don't do actual requests. But if there is no cache, execute the fetch as usual. Now you can see how with this, there is a huge potential to implement caching like this on the client-side. But this got me thinking, what if instead of cached responses, we could return mock responses? Let's try that out. So the same worker, but this time in the fetch event, we're responding with this hard-coded response body and status code 200. And now whichever request our app makes is going to leave the app and it's going to hit the worker, and in response, this static response will be returned. This is pretty great, so let's just use service workers for mocking, and the problem is solved. But if you try to do that, you'll be faced with a number of challenges. First of all, service workers have limited execution context, and this is mainly due to security considerations. You cannot access the window object or DOM, and obviously your client-side code and your functions, utilities, and libraries. The worker can get out of date, and each time you register a worker, this does not necessarily push the latest worker to the browser, and you may end up with an obsolete worker, which is not nice. When you hard-load the page, the worker will stop, and you would have to reload the page to see it running again. This creates a distorted developer experience. Workers also control unrelated clients, and they do it because the worker establishes connection with a client based on its URL. So you may open a completely different project on the same URL, on localhost, and see an unrelated worker kicking in and trying to mock responses, and this is just confusing. Despite those challenges, I like the idea, and a couple of years back, I wrote a library that's called MockServiceWorker, or MSW for short, and it leverages the ability of service worker to intercept requests and does that gracefully. MSW is the closest thing to an actual server without having to create one, and this is thankfully to service workers. Let me show you how the workflow with the library looks like before we dive into internals. So first things first, I'm going to tell the library which requests to capture, and I'm writing these things called request handlers, which are just functions that described request information and which response to produce. Here I'm targeting a REST API request, which is a get request to slash user end point, and I'm producing this mocked response as JSONBody with first name equals John. Once I'm done with the handlers and I described all the server behavior, I'm creating a worker by calling setup worker and providing my handlers and then starting this worker, which is going to register and activate the worker in the browser. I've stored this module in my app, and I can see that now when I make a request to slash user, it gets intercepted and I receive the mock response. But how exactly does it work? So whenever the app makes a request, it gets intercepted by the worker. The worker clones the request and sends it back to MSW because MSW doesn't execute on the worker. Instead, it runs next to your client code, so you can use your favorite languages, libraries, and utilities there. MSW tries to find the handler for this request. When it does, it returns the mock response to the worker, and worker uses it to respond to the original request in the client. Now with MSW and due to service workers, there is no more of this ghost request, requests that never happen. Instead, you can clearly see requests in the network tab because they actually happen, the same as in production. There is no more request clients tabbing. We don't meddle with fetch or other libraries, and because of that, we're getting full specification compliance because request client is called and response is composed on the service worker side. If any of these parts is invalid, an exception will be thrown. And there's also no more conditional URLs, which is, again, because of service worker, and we can get identical application behavior, the same as our app does in production. So when would you use MSW? Well, first of all, you can use it for testing. Once you've written the request handlers, you can reuse them for integration tasks and also for end-to-end tasks with Cypress or Puppeteer. You can push this concept even further and use MSW for development. Let's say your backend is not ready or it undergoes a change, or you want to experiment with some third-party API. You can just try it out with MSW first. The next one is debugging, and that's my favorite because whenever there's a data-related issue, you can just create a handler for a specific request and try to match what response crashes your app. Once you do, you have a reliable way to reproduce the issue and thus to fix it. You also can use MSW for prototyping. When you're building your next awesome product, the backend may not be ready, but you can still write client-side code, and you can use MSW to act like a backend. So here we can try RESTful API today, and maybe tomorrow we want to have a peek at how our application would feel with GraphQL. So we just use GraphQL request handler, and we don't have to commit to the entire ecosystem of GraphQL just to try it out. There are much more features that MSW offers. For example, it's written in TypeScript, so you can use type definitions to annotate your requests and responses and make sure that you're safe. It's request-client agnostic, so you can use MSW with any request-issuing client, be it native fetch or React query or Apollo. There is no library-specific logic there. You can also use MSW with Node.js, so to run it in Jest, or when you develop in Node.js servers to mock third-party communication. Companies like Google, Spotify, and Gatsby are already using MSW, and you are one command away from trying it yourself, so don't miss out. Make sure to check out the repo at mswjs.msw and also the docs at mswjs.io where you can read more about the library, its API, see some usage examples, and also check out the comparison with other API mocking tools out there. If you're a more hands-on learner, definitely look at that YouTube video to mock a basic HTTP response that goes from start to finish, configuring MSW and getting the mocks ready in under four minutes. And also don't miss Epic React course by Ken C. Dots, which is a huge collection of material on how to build awesome React apps, and it features MSW in its testing section. Make sure to follow API Mocking on Twitter to stay in touch with releases and library updates, and you can also follow me just to say hi or ask a question. I'm always happy. I hope this talk was useful to you, and now you can write more confident tests, and perhaps MSW can contribute to that. Thank you. Maybe in the meantime, you would like to tell us something about what's going on with MWS. No, sorry, MSW. That's the right order. I heard there are some nice things brewing. Yeah, definitely. We got a few people joining to the team, and with more folks using it and contributing, we get a lot of requests, and we're now working on improved GraphQL integration so we can support subscriptions, and we're also pushing for WebSockets because, well, we use WebSockets, but there are not many solutions that support mocking that. Yeah. And we're trying to also support more maybe approaches in development and testing you have. For example, I know that folks prefer a data-driven approach, so they can define a schema, and then they can generate different entities and have the mocks based on that. We're also working on the data modeling solution to help out. That sounds awesome. So the community is really growing. Yeah, it's crazy. Maybe for it. Okay, cool. I see there are some questions coming in. Stephen wants to know, could you integrate MSW and PECT? Do you have any info on that? I haven't tried that, but I don't think YU wouldn't be able to, so I guess it's a cool opportunity to try it out and let us know. Okay, cool. So Stephen, if you try it out, please let us know what the answer is. Steve wants to know, can you enumerate some differences between Cypress mock feature and MSW? I guess that by Cypress mock, they mean like Cypress intercept, perhaps. I haven't heard about the mock API. But in any case, I think that in Cypress, request interception is done through stubbing of fetch or XHR. So that would be the main difference, because when you use service workers, you don't have to do that, like I mentioned in the talk. You actually allow your app to make the requests, and everything happens. All the request client code is called, and then the service worker intercepts it in the network level. So I suppose that would be the main difference. But you can use MSW with Cypress as well and have it running alongside your app in Cypress, and then have it handling the mocking. Okay, cool. I think that also maybe answers the next question. Vatilis wants to know, would you use MSW in production as some kind of API gateway? Well, there are a few people who use MSW in production, but it's mainly for educational purposes. I don't think that it's a good idea, because once you push the worker to your user's machine, you would be responsible for keeping it in sync. And as I mentioned, workers are quite tricky. And even if you register the new version, it doesn't mean that the old one will get replaced immediately. And if you don't want to get stuck in this out-of-date mocks logic, I would probably not recommend doing so. But you can definitely try. I mean, service workers are meant for production. And if you find value in that, in running MSW in production, perhaps give it a try. So it's possible, but debugging is maybe a more common use case. Yeah, it mainly focuses on testing and development rather than acting like a gateway. Okay. Richard says, hi, was just about to ask about WS-MOX. When is that likely to be introduced? Will it support dynamic responses? I'm not sure I follow WS-MOX. That's what was written. Maybe Richard can clarify on that. And in the meanwhile, let's go to Testibyte's question. What are some of the use cases for mocking RP? Mocking third party RPs makes sense almost always. Would you mock RPs you own? Well, I guess it depends on the kind of testing level you are on. Because if you're testing your components, like the integration of them, like let's say some login form or checkout form, just a little piece of code you wrote. But it already has multiple components and perhaps it does requests. So I guess it makes sense to mock your own API here. Because you don't want to rely on an actual server during this test. And once you bring it one level up to end-to-end tests, then probably here you want to hit your actual endpoints and make sure that it works for customers. I would probably split it like that. Yeah, it probably depends on the size of your project and what level of test you are doing. OK. Have we heard back from Richard yet so that we can answer the question? Oh, WebSocketMocks. Sorry, he meant WebSocketMocks. OK, got it. Got it. Sorry. Do you want me to repeat the whole question? Yeah, if you don't mind. Hi, I was just about to ask about WebSocketMocks. When is that likely to be introduced? Will it support dynamic responses? OK. So, yeah, as I mentioned, we are working on WebSocket support as well. I kicked off the prototype a few months back, but I guess it needs still some polishing. And I would expect it somewhere this year, but probably closer to summer. It depends on the priorities, but it would definitely be nice to have it there so people can try it out and let us know, because we obviously can't predict all the use cases that you have. So once it's out, try it out and let us know. And in regards to dynamic responses, I'm actually not sure. I don't have much experience with WebSockets myself, using them. I'm digging through how they're implemented and specification, but I haven't got to the part of this feature yet. So perhaps if you're concerned with this, I would suggest you go to GitHub and just raise this in the pull request. So we won't miss it out. Thanks. Well, summer is not that far away. So can we try it out soon? We'll do our best. Do you have any plans when the next version is going to come out? I mean, we already talked a little bit about the rest of the roadmap. Yeah, we have a few small developer experience improvements, which I think kind of lands next week. A lot of people were asking to help them debug scenarios when requests are not intercepted. And this is really common. Sometimes there is a typo, sometimes there is a wrong domain name or something. And we already have some basic logic that's just going to tell you, hey, this is not intercepted, but we're going to push it further and we're going to use some string matching logic that's going to suggest to you the handler that you perhaps meant. So whenever you request the resource, we're going to check, hey, maybe you did a typo or maybe you requested a GRFQL query, but you meant imitation with the same name. So expect that, I think, next week. That's such a great and helpful feature. So next week might be the exact perfect time to try this framework out if one hasn't already. So I got to definitely put that on my own list. And Tiger wants to know, will WebSockets enable GRFQL subscriptions with MSW? Yeah, yeah, definitely. I hope so. Last time I checked, they do work through WebSockets. So once we have WebSockets rolled out first, we're going to look into GRFQL subscriptions and if we need to do any extra work there. OK, awesome. And I hear people are really looking forward to that feature. So that's also coming next week or? You mean what, WebSockets? Yeah. Well, that's already part of part of the framework. Oh, no, no, no. It's the thing I meant that's going to happen close to summer. So they'll need some work. OK, great. We don't have any more questions right now. So if there's anything, anything else. Yeah. Thank you for the questions. It's been really great. It was great having you here and see you in the in the speakers room, please. If there is anybody else you want to talk about with Artem, join on special chat. And thank you so much, Artem. Yeah, thank you for having me. It's been a pleasure. Thanks. Bye.
