Testing Mail Service With Playwright

Rate this content
Bookmark

We send emails to our users - account verification and newsletters. We allow the user to contact us by sending an email via inbuild form. Do we? Does the user receive an account verification email or exactly what notification they signed up for? We can cover this functionality as part of E2E tests: get an email and open it to check what is in it. We will need Playwright and a fake SMTP server to capture emails sent by the app.

Kat Kmiotek
Kat Kmiotek
17 min
03 Nov, 2022

Comments

Sign in or register to post your comment.

Video Summary and Transcription

This Talk discusses how to test mail service with Playwright, covering e-mail verification, reset password user journey, and more. It explores the use of third-party providers for reliable e-mail delivery and demonstrates how Playwright can help perform checks on e-mail content. The Talk also introduces the concept of a fake SMTP server and showcases how fixtures can be used to access the SMTP server and perform assertions on the HTML body of emails. Playwright's HTML rendering feature allows for interaction with email content as if it were a regular web page. It highlights the ability to render HTML from API calls, perform assertions on the rendered page, and exclude dynamically generated data from visual regression tests.

Available in Español

1. Testing Mail Service with Playwright

Short description:

In this talk, I will show you how to test mail service with Playwright. We will cover e-mail verification, reset password user journey, and more. By using third-party providers like Amazon SES, MailChimp, Postmark, or SendGrid, we can ensure reliable e-mail delivery. However, the problem lies in what gets delivered to the user. Today, I will demonstrate how Playwright can help perform checks on e-mail content and ensure a seamless user experience. Additionally, I will introduce the concept of a fake SMTP server, which captures and stores e-mails in a test environment. Tools like Mailhug provide a user-friendly interface for manual verification and comprehensive API documentation.

Hi, everyone, my name is Kat Kniotek, I work as a quality engineer at Zoopla, and thank you for joining my talk today. I will be talking about testing mail service with Playwright. I will show you how easy it is to add e-mail verification or reset password user journey to your end-to-end tests. And honestly, I really like how clever you can go here with Playwright. So let's start.

Depending on the application you are working on, you may or you may not communicate with the user by the e-mail. But if you do, you would send them newsletters, order confirmations. If you work on the e-commerce website, reset password link if the action is requested by the user, e-mail verification request and any sort of receipts or notification they may ask for. And you usually use the third party providers to handle this, may transportation for you. And those would be Amazon SES, MailChimp, Postmark and SendGrid, just to name a few. And it's a really good idea because we can rely on them to make sure that e-mail is actually delivered. But the problem statement of this talk is what gets delivered to the user. Can they click the link and reset their password? Does verification e-mail actually take them to the correct domain? If body content of the e-mail actually matches our design system, and of course, if by mistake, we don't send any sensitive data or any random data like this test as a subject of the e-mail. So today, I will show you how all those checks can be performed with help of playwright. So let's start.

So to describe the e-mail communication in a production application, you will have application communicating with third party service, and third party service will be responsible for delivering e-mails to the user or user groups. In a test environment for this demo, instead of sending requests to the third party service, we'll be sending e-mail to fake SMTP service and then from there, we will be able to access data from playwrite tests. OK, I just introduced a new thing, a fake SMTP server, what it is. You can think about it as a container that will be capturing and storing all e-mails sent by application in a test environment. And for this purpose, you can choose one of many tools available. I listed just a few here. Some of them are paid and some of them are open source. It's up to you which one you will use. For this demo, I'm using Mailhug. All of them usually come with really nice graphical user interface that you can access in your browser that looks exactly like the mailbox. You can browse e-mails, you can delete e-mails, store attachments or forward e-mails. It's great for manual verification of what gets sent to the user. That sort of fake SMTP server will also come with great documentation about their API. So with Swagger. Swagger would list all available end points on the service with methods that are available and how the payload should look like and what you expect to see as a response of the API call to the server.

2. Integrating Mailhook API with Playwright Testing

Short description:

Here's the example call to the Mailhook API, querying for emails sent to the test user. The response includes the status, number of emails sent, and the email objects with ID, from, to, content, and body. We can perform assertions on the email content, including checking for specific strings, elements, and styling. To translate this API testing to Playwright tests, we need to access the SMTP server, retrieve the most recent email, and perform assertions on the HTML body. Playwright's HTML rendering feature allows us to interact with the email content as if it were a regular web page. We can click links, check for elements, and verify URLs. To start, we'll access the SMTP server from our end-to-end tests using fixtures, which provide isolated contexts with separate configurations.

Here's the screenshot of the example call to the Mailhook API. So as a search point I'm querying here. Based on their documentation, I provided query params to get all emails that has been sent to the test user at example.com and on the right-hand side, you can see how the response looks like. So good, because status is 200. And then you can see how many emails have been sent to this user, so the service endpoint returns three.

Brilliant, because I was just testing a little. And all of them are listed as email objects in the items array, so on line five, and each individual email object would have ID, from, to, content, and body. And if you look closer, below line 34, well, from line 34, you can see that body is actually HTML body, HTML code. So again, here we can do some assertions as well. We can check if the certain strings are present, if the certain elements are present, let's say button or link, but also verify if styling, right, styling has been applied, right? Just to mention, I'm using Thunder Client here. So, tools similar to the Postman or Insomnia, if you're familiar with them.

A fake SMTP server comes with some nice features allowing manual testing, but we'll use the same SMTP server in our play write tests, so let's have a look how to translate what I just showed with, let's say, API testing to play write tests. Let's start with pseudo coding, what do we want to do? So, first, our test will need to have precondition and that depends on your application. That could be the action that triggers email verification to be sent. Let's say, user creates the account or using completes the order and order confirmation gets sent and so on. Because I'm not connecting to any real application here, it was just a script that triggers sending email.

Okay, so what do we want to focus in the test? We want to do exactly what I did manually with connecting to the SMTP server. So, I want to access SMTP server, I want to get most recent email that has been sent to the user. So, in a response there was items array, so I want to get the first element of it, so with index zero. And I want to get an HTML body part, because that's where I want to do assertions on. And the great magic feature of Playwright, you can actually use this HTML body to render a new page and do the same assertions as you would do on a regular page in a browser. So, you can click verify button link, you can check if certain elements are present and you can also verify if by clicking the verify button, you will open a new tab and you can do assertion if the new tab which is also page, will have expected URL. Okay, so, that's a lot to do. So, where to start? I will start with accessing SMTP server from my end-to-end tests. And for this purpose, I will be using fixtures. What fixtures are? So, probably in your application, you have Playwright Config. Well, in your end-to-end tests, you have Playwright Config and you specify base URL value there. And that would be fronted of your application client code, the base URL of your old tests. But you can specify also fixtures, so like isolated contexts that will be used in a certain test. So, here I'm creating the fixture called email API and that one will have separate base URL value, extra HTTP headers and so on.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Scaling Up with Remix and Micro Frontends
Remix Conf Europe 2022Remix Conf Europe 2022
23 min
Scaling Up with Remix and Micro Frontends
Top Content
Do you have a large product built by many teams? Are you struggling to release often? Did your frontend turn into a massive unmaintainable monolith? If, like me, you’ve answered yes to any of those questions, this talk is for you! I’ll show you exactly how you can build a micro frontend architecture with Remix to solve those challenges.
Full Stack Components
Remix Conf Europe 2022Remix Conf Europe 2022
37 min
Full Stack Components
Top Content
Remix is a web framework that gives you the simple mental model of a Multi-Page App (MPA) but the power and capabilities of a Single-Page App (SPA). One of the big challenges of SPAs is network management resulting in a great deal of indirection and buggy code. This is especially noticeable in application state which Remix completely eliminates, but it's also an issue in individual components that communicate with a single-purpose backend endpoint (like a combobox search for example).
In this talk, Kent will demonstrate how Remix enables you to build complex UI components that are connected to a backend in the simplest and most powerful way you've ever seen. Leaving you time to chill with your family or whatever else you do for fun.
Making JavaScript on WebAssembly Fast
JSNation Live 2021JSNation Live 2021
29 min
Making JavaScript on WebAssembly Fast
Top Content
JavaScript in the browser runs many times faster than it did two decades ago. And that happened because the browser vendors spent that time working on intensive performance optimizations in their JavaScript engines.Because of this optimization work, JavaScript is now running in many places besides the browser. But there are still some environments where the JS engines can’t apply those optimizations in the right way to make things fast.We’re working to solve this, beginning a whole new wave of JavaScript optimization work. We’re improving JavaScript performance for entirely different environments, where different rules apply. And this is possible because of WebAssembly. In this talk, I'll explain how this all works and what's coming next.
Debugging JS
React Summit 2023React Summit 2023
24 min
Debugging JS
Top Content
As developers, we spend much of our time debugging apps - often code we didn't even write. Sadly, few developers have ever been taught how to approach debugging - it's something most of us learn through painful experience.  The good news is you _can_ learn how to debug effectively, and there's several key techniques and tools you can use for debugging JS and React apps.
Testing Web Applications with Playwright
TestJS Summit 2022TestJS Summit 2022
20 min
Testing Web Applications with Playwright
Top Content
Testing is hard, testing takes time to learn and to write, and time is money. As developers we want to test. We know we should but we don't have time. So how can we get more developers to do testing? We can create better tools.Let me introduce you to Playwright - Reliable end-to-end cross browser testing for modern web apps, by Microsoft and fully open source. Playwright's codegen generates tests for you in JavaScript, TypeScript, Dot Net, Java or Python. Now you really have no excuses. It's time to play your tests wright.
Full-Circle Testing With Cypress
TestJS Summit 2022TestJS Summit 2022
27 min
Full-Circle Testing With Cypress
Top Content
Cypress has taken the world by storm by brining an easy to use tool for end to end testing. It’s capabilities have proven to be be useful for creating stable tests for frontend applications. But end to end testing is just a small part of testing efforts. What about your API? What about your components? Well, in my talk I would like to show you how we can start with end-to-end tests, go deeper with component testing and then move up to testing our API, circ

Workshops on related topic

Detox 101: How to write stable end-to-end tests for your React Native application
React Summit 2022React Summit 2022
117 min
Detox 101: How to write stable end-to-end tests for your React Native application
Top Content
WorkshopFree
Yevheniia Hlovatska
Yevheniia Hlovatska
Compared to unit testing, end-to-end testing aims to interact with your application just like a real user. And as we all know it can be pretty challenging. Especially when we talk about Mobile applications.
Tests rely on many conditions and are considered to be slow and flaky. On the other hand - end-to-end tests can give the greatest confidence that your app is working. And if done right - can become an amazing tool for boosting developer velocity.
Detox is a gray-box end-to-end testing framework for mobile apps. Developed by Wix to solve the problem of slowness and flakiness and used by React Native itself as its E2E testing tool.
Join me on this workshop to learn how to make your mobile end-to-end tests with Detox rock.
Prerequisites- iOS/Android: MacOS Catalina or newer- Android only: Linux- Install before the workshop
Using CodeMirror to Build a JavaScript Editor with Linting and AutoComplete
React Day Berlin 2022React Day Berlin 2022
86 min
Using CodeMirror to Build a JavaScript Editor with Linting and AutoComplete
Top Content
WorkshopFree
Hussien Khayoon
Kahvi Patel
2 authors
Using a library might seem easy at first glance, but how do you choose the right library? How do you upgrade an existing one? And how do you wade through the documentation to find what you want?
In this workshop, we’ll discuss all these finer points while going through a general example of building a code editor using CodeMirror in React. All while sharing some of the nuances our team learned about using this library and some problems we encountered.
Testing Web Applications Using Cypress
TestJS Summit - January, 2021TestJS Summit - January, 2021
173 min
Testing Web Applications Using Cypress
WorkshopFree
Gleb Bahmutov
Gleb Bahmutov
This workshop will teach you the basics of writing useful end-to-end tests using Cypress Test Runner.
We will cover writing tests, covering every application feature, structuring tests, intercepting network requests, and setting up the backend data.
Anyone who knows JavaScript programming language and has NPM installed would be able to follow along.
Build a powerful DataGrid in few hours with Ag Grid
React Summit US 2023React Summit US 2023
96 min
Build a powerful DataGrid in few hours with Ag Grid
WorkshopFree
Mike Ryan
Mike Ryan
Does your React app need to efficiently display lots (and lots) of data in a grid? Do your users want to be able to search, sort, filter, and edit data? AG Grid is the best JavaScript grid in the world and is packed with features, highly performant, and extensible. In this workshop, you’ll learn how to get started with AG Grid, how we can enable sorting and filtering of data in the grid, cell rendering, and more. You will walk away from this free 3-hour workshop equipped with the knowledge for implementing AG Grid into your React application.
We all know that rolling our own grid solution is not easy, and let's be honest, is not something that we should be working on. We are focused on building a product and driving forward innovation. In this workshop, you'll see just how easy it is to get started with AG Grid.
Prerequisites: Basic React and JavaScript
Workshop level: Beginner
0 to Auth in an Hour Using NodeJS SDK
Node Congress 2023Node Congress 2023
63 min
0 to Auth in an Hour Using NodeJS SDK
WorkshopFree
Asaf Shen
Asaf Shen
Passwordless authentication may seem complex, but it is simple to add it to any app using the right tool.
We will enhance a full-stack JS application (Node.JS backend + React frontend) to authenticate users with OAuth (social login) and One Time Passwords (email), including:- User authentication - Managing user interactions, returning session / refresh JWTs- Session management and validation - Storing the session for subsequent client requests, validating / refreshing sessions
At the end of the workshop, we will also touch on another approach to code authentication using frontend Descope Flows (drag-and-drop workflows), while keeping only session validation in the backend. With this, we will also show how easy it is to enable biometrics and other passwordless authentication methods.
Table of contents- A quick intro to core authentication concepts- Coding- Why passwordless matters
Prerequisites- IDE for your choice- Node 18 or higher
Building out a meaningful test suite that's not all E2E
TestJS Summit 2023TestJS Summit 2023
89 min
Building out a meaningful test suite that's not all E2E
Workshop
David Burns
David Burns
We're all taught to follow the Testing Pyramid but the reality is that we build out the Testing Christmas Tree. In this workshop, David will talk you through how to break down projects and put the tests where they need to be. By the end of the workshop you will be able to update your projects so that anyone and everyone can start contributing and truly living up to "Quality is everyone job".
He will walk you through:- Component Testing- API Testing- Visual Regression Testing- A11Y testing
He will also talk you through how to get these all setup in your CI/CD pipeline so that you can get shorter and faster feedback loops.