Testing Mail Service With Playwright

♪♪♪ ♪♪♪ ♪♪♪ Hello, everyone. My name is Kat Kmiotek. I work as a quality engineer at Zoopla. And thank you for joining my talk today. I will be talking about testing mail service with Playwright. I will show you how easy it is to add email verification or reset password user journey to your end-to-end tests. And honestly, I really like how clever you can go here with Playwright. So let's start. Depending on the application you are working on, you may or you may not communicate with the user by the email. But if you do, you would send them newsletters, order confirmations if you work on the e-commerce website, reset password link if the action is requested by the user, email verification request, and any sorts of receipts or notification they may ask for. And you usually use the third-party providers to handle this mail transportation for you. And those would be Amazon SES, MailChimp, Postmark, and SendGrid, just to name a few. And it's a really good idea because we can rely on them to make sure that email is actually delivered. But the problem statement of this talk is what gets delivered to the user. Can they click the link and reset their password? Does verification email actually take them to the correct domain? If body content of the email actually matches our design system? And of course, if by mistake we don't send any sensitive data or any random data like this test as a subject of the email. So today, I will show you how all those checks can be performed with help of Playwright. So let's start. So to describe the email communication in a production application, you will have application communicating with third-party service. And third-party service will be responsible for delivering emails to the user or user groups. In a test environment for this demo, application, instead of sending requests to the third-party service, will be sending email to fake SMTP service. And then from there, we will be able to access data from Playwright tests. Okay, I just introduced new thing, fake SMTP server. What it is? You can think about it as a container that will be capturing and storing all emails sent by application in a test environment. And for this purpose, you can choose one of many tools available. I listed just a few here. Some of them are paid and some of them are open source. It's up to you which one you will use. For this demo, I'm using Mailhook. All of them usually come with really nice graphical user interface that you can access in your browser. That looks exactly like the mailbox. You can browse emails, you can delete emails, store attachments or forward emails. It's great for manual verification what gets sent to the user. And that sort of fake SMTP server, it will also come with great documentation about their API, so with Swagger. Swagger would list all available endpoints on the service with methods that are available and how the payload should look like and what you expect to see as a response of the API call to the server. Here's the screenshot of the example call to the Mailhook API. So as a search point I'm querying here. Based on their documentation, I provided query params to get all emails that have been sent to the test user at example.com. And on the right-hand side, you can see how the response looks like. So good, because status is 200. And then you can see how many emails have been sent to this user. So the service endpoint returns three. Brilliant, because I was just testing a little. And all of them are listed as email objects in the items array, so on line five. And each individual email object would have ID, from, to, content and body. And if you will look closer below line 34, well, from line 34, you can see that body is actually HTML body, HTML code. So again, here we can do some assertions as well. We can check if the certain strings are present, if the certain elements are present, let's say button or link, but also verify if styling, right styling has been applied. Just to mention, I'm using Tandr client here. So tools similar to the Postman or Insomnia, if you are familiar with them. Okay, so fake SMTP server comes with some nice features, allowing manual testing, but we'll use the same SMTP server in our playwright test. So let's have a look how to translate what I just showed with, let's say, API testing to playwright test. Let's start with pseudocoding, what we want to do. So first our test will need to have precondition. And that depends on your application. That could be the action that triggers email verification to be sent. Let's say user creates the account or using completes the order and order confirmation gets sent and so on. Because I'm not connecting to any real application here, it was just a script that triggers sending email. Okay, so what do we want to focus in the test? We want to do exactly what I did manual with connecting to the SMTP server. So I want to access SMTP server. I want to get most recent email that has been sent to the user. So in a response, there was items array, so I want to get the first element of it. So with index zero. And I want to get HTML body part because that's where I want to do assertions on. And the great magic feature of Playwright, you can actually use this HTML body to render new page and do the same assertions as you would do on a regular page in a browser. So you can click verify button link, you can check if certain elements are present, and you can also verify if by clicking the verify button, you will open new tab and you can do assertion if the new tab, which is also a page, will have expected URL. Okay, so that's a lot to do. So where to start? I will start with accessing SMTP server from my end-to-end tests. And for this purpose, I will be using fixtures. What fixtures are? So probably in your application, you have Playwright config. In your end-to-end tests, you have Playwright config. And you specify base URL value there. And that would be fronted of your application client code, the base URL of your old tests. But you can specify also fixtures, so like isolated contexts that will be used in a certain test. So here I'm creating the fixture called email API. And that one will have separate base URL value, extra HTTP headers, and so on. So that fixtures allows you to add maybe different end point tests from the same repository test, also backend or other services like here, example, Mailhook. So as you can see, I provide the base URL value, which is running on localhost Mailhook server. I pass authorization as extra HTTP headers. And I also specify to ignore HTTP errors as true, because I'm trying to connect to localhost. And this API context that is defined on line 11 will be used on all calls made to email API. Okay, and how it looks like actually when you use it. So on line 14, here I can see that, yeah, on the left-hand side, you probably don't see this 14. It's 14, line 14. I do exactly the same what I did in the Tandr client, where there was manual testing end point. So on the API context that I just defined, I use the get method to query the search end point. For the query params, the same as in the Tandr client example. So I query all the emails that has been sent to the test user. And then I return the response. And here, this may look a little bit complex. What is happening here? Here on line 26, I get the first email, so items index one, content and body, to get the HTML body of the most recent email that has been sent to the user. What is happening on line 26 might look a little bit confusing. So when I was showing in the Tandr client how the body looks like, HTML body of the email, maybe you noticed that there were appendant equals 3D strings in some places. This is because the type of encoding used for email transportation called quoted printable. So here on line 28, I just tidy up the email body from this encoding. It looks complex, but I rely on helpers libraries to do it for me. And then at the end, I just return the HTML body. So now, this HTML body can be used to render the page. So here in the email page object, I define my custom go to method that takes as a parameter actually the HTML code from the email. And on line 14, I instruct playwright. So this time I would be going to forward slash email URL. I want this route request to be fulfilled with the email body. So then when on line 16 actually I go to this URL, what will be rendered in the browser will be actually email body. And then I will be able to interact with it as I would do with any page. So I will be clicking verify button on line 22 and wait until new tab will be open and will return the new tab. Putting it all together on line 15. But basically that's how the test will look like. So obviously first precondition. But then, as I said, I specify the separate fixture. So that's how I imported it to the particular test email API on line 15. So then I use the email API to get email body on line 18 for the test user. The test user that I use for manual verification as well. And then once I had this HTML body on line, I think it's line 12, I can use this HTML body to render the email page. So I click the button on this email page and then assert that actually URL of this newly opened tab matches test.js summit. Okay. That may work, may not. But if I will run the test actually in a debug mode, I can see browser opening and play around inspector opening beside it so I can step through the test and see that actually works. So what is happening here? That's the actually first page that opens because the API call happens in seconds just before the browser opens. And here we are at the step that I go to forward slash email and the email is actually rendered. And here, player attempts to click element button with ID button one. And once that one is successful, the new tab is open. And as you can see the URL is exactly the one that we expected to appear here. So when I seen it for the first time, I was like, whoa, that's amazing. You can actually get HTML from the API call and use this HTML to render the page. Yeah, it's amazing. And going from here, you can think how many other assertions you could do on the actually rendered page. So you can do visual regression tests. So do the screenshot comparison to make sure that nothing has changed in the version of the email that you are sending now comparing to the baseline image. Here you may say, OK, I sent to the user as a maybe title or I don't know, header of the email I rendered their username or their first name or initials. And that will be different with every test run because you dynamically generate test data. So here with Playwright API, you can actually exclude this area from the comparison. So let's say mask the h1 locator and that will help you with the testing the email if the data is dynamically generated. You can also, as I mentioned, once we have this email rendered as a page, you can do all sorts of assertions like as you would do on a regular page. So let's say that check that verify button actually has the correct attribute or if the state is changing when you hover over and so on and so on. So I hope that I showed how the Playwright can be used for testing mail, email body and what actually gets sent to the user. So I checked that email got delivered to the user. I checked functionality of the button. So you could see that action click has been successful, that the navigation happened to the correct URL. So with Playwright you are also able to do a visual check on the email, on the content body and obviously all other assertion on DOM elements as you would do. Okay, that's the end of the talk. I hope that you found it useful. All the links that I just posted here will help you to maybe try and build a similar solution and apply it to your tests. As I mentioned, I was using Mailhawk. So I provided a link to Mailhawk Swagger, Mailhawk Docker image with OLF, so you can just run it locally. More explanation about equal 3D. So the Quotent printable encoding. A few words about Tundr client, so my go-to tool for manual API tests. And link to contact with me if you would have any questions. So yeah, I just look forward to hear your questions now. If you want to access the presentation, you can just scan the QR code and that will take you to the slides. Thank you. ♪♪♪