Writing Good Tests for Vue Applications (e2e vs. Component Tests, Features of a *Good* Test)

Hello, London. How are you doing? Awesome. So yeah, today I talk about testing. I will start with a little story.

A couple of years ago, my team and I were tasked to work on a completely new project so we could start a new project from scratch. And this is good news for us developers because most of us like to start from scratch with new projects. And we were a bunch of very motivated people and we wanted to do everything the right way. And the right way for us also included writing tests. So we wrote a lot of them. We wrote a lot of end-to-end tests. We wrote a lot of component tests and the years passed by and everything turned out great. And of course that's not how it went unfortunately. So what was the problem? The problem was that although we wrote a lot of tests, I, at that point, didn't know a lot about how to write good tests. So I was new to this testing topic and I made some mistakes. Let's put it that way. So a lot of those promises that come with advocates for testing always tell us didn't come. We still had a big mess of a code base in the end. But what are those promises that good tests should enable us to do? And the first one is good tests enable us to refactor our code with confidence. And also, good tests allow us to write better code faster. And good tests allow us to deploy on Friday if we want to. We still don't have to. So, I said the phrase good tests a couple of times. And what do I mean by good tests? How do we write good tests? And the good news is there are only two things, two very important things you have to consider. The first one is write test first. And the second one is decouple all the things.

Let's start with the second one, decouple all the things. What do I mean by all the things? And there are three aspects in which we can decouple our tests from our code base. The first one being we can decouple our tests from the test framework we are using. And now you might wonder why should we do this? Why should we go the route of decoupling our tests from the test framework? What's the point of it? And the thing is, imagine you are using, you have hundreds of tests and you're using a kind of old test framework and the maintainers of the test framework decide they don't want to maintain this framework anymore. And now you suddenly have to rewrite hundreds of tests and this can be a lot of work.

Or there's a new and better test framework out there like just now in the Vue ecosystem or generally in the ecosystem we have V-Test which is very popular in the Vue ecosystem but tends to replace Jest maybe. And if you decouple your test from the test framework, you can switch frameworks just by rewriting of lines of code instead of all of your tests.

So what's the problem and how can we fix this? Here we can see a test for a simple application and because we are using this test for all the other, this application for all the other examples as well I want to show you this simple application. It's a very straightforward shopping list application and we can add something like bread for example and then we have bread on our shopping list and we can add something like milk and then we have milk on our shopping list and in the end we want to also be able to remove those items by clicking on those items. For now this doesn't work. We will add this feature later.

So this is the simple application, we test and in this test, this test is coupled to the test framework. In this instance Cypress for example. So you see this by this Cypress, Cy prefix and we are using methods from Cypress and we are using a get method to get some selector and typing something in it and we use again we use the get method of Cypress to click some buttons and so on. So how can we improve this? How can we decouple this test from Cypress? From the test framework? So we can switch to playwright for example just by changing 1 file instead of hundreds of tests. And the fix can look something like this. So here you see now we inject a generic driver so in the test callback we inject a generic driver, implementation on a generic driver object and for the test itself it doesn't matter if this driver is a playwright driver, a Cypress driver or maybe even a V test driver, it just doesn't matter. And then instead of the Cypress object we use the driver object which has a particular interface but it doesn't matter which implementation is behind it. Again, we have this get method to get some selectors and do some stuff like clicking or typing text. So this is for end-to-end tests. That way we can decouple end-to-end tests. Now you might wonder what about component tests. And for component tests I think this aspect isn't that important because for component tests we are closer to the code and we have to use specific methods and we have to work with return variables and stuff like that. And implementing a truly generic driver for component tests is harder to do, and I think it's also not that important. But what we still can do is to move all the test methods we use in our tests, like expect it and mount for example into our own utils files. And this enables us to swap test frameworks very easily. Imagine you were using Chess, for example, and now you want to switch to VTEST. If you have all your files or all your methods in this utils file, you can simply change this utils file and all your tests will still work. And the second thing this enables us to do is that, for example, we can wrap the mount function. And imagine you're having a UX Store or something like that or a router, and you can create your own custom implementation of the mount function which always loads your router or your OpinIO Store and so on. So much for decoupling for end-to-end tests and component tests. The next way in which we can decouple our tests from the code in this case is by decoupling our tests from implementation details. And you maybe heard of that already because it's probably one of the most important aspects when it comes to decoupling our tests. And let's take a look at the problem first again. So we see the test from before.

And although it's now decoupled from the framework, it's still coupled to implementation details. And one of the most common ways for end-to-end tests, how they are coupled to implementation details of our code, is by coupling the tests to CSS selectors. Like in this instance, we coupled this test to the item input CSS selector. And whenever some developer wants to change the name of the CSS selector or wants to change or move the CSS selector somewhere else, the CSS class, then this test breaks, although it doesn't have to break, because the functionality doesn't break.

And what our tests are supposed to do is to make sure the functionality works correctly and not if some CSS selector is in some place and some element. So this test is coupled to one particular implementation detail, CSS selectors. And how can we fix this? There are multiple ways how to fix this, but in my opinion, the most or the best way how to fix this is by using semantic selector methods. Like in this case, we have this find by label text method. And so we can make sure that not only we don't couple our test to CSS selectors, so to implementation detail, but also make sure that we have some base level of accessibility ensured, because by this find by label text method, we ensure that an input field has a label, which it should have for accessibility. So we ensure this field has a title label, and then we type something in this title label field.

Another semantic selector method is find by role, which we can use for things we can click mostly, in this case, for a button. And again, we not only decoupled our test from CSS selectors, but also ensure some base level accessibility by making sure that we use a button, instead of a diff or something like that. So here we use find by role button, and then we select the add item button and click it. Last but not least, we have this find by text method, where we can make sure that we basically test our application through the eyes of our users, which don't watch for some CSS selector, which they don't even see, but they watch for some text being on the screen or not being on the screen.

For component tests, it is very important to decouple them from implementation details. And here, we see a component test that is coupled to implementation details in the worst possible way, and we already see this in the description, where we write, it should emit a delete event when executing the delete method. So this executing the delete method, is the problem here. And we see it in the test also here we directly execute the delete method on some component. And this is the worst form of coupling our tests to implementation details, our component tests.

So how can we fix this? And again, we can fix this by writing our tests more from the perspective of a real user, how a real user would use our application. And what I do here is using the wonderful testing library. So there's a package out there named testing library. And with that, we get those semantic selectors we saw previously and also some other helpers like you can see here, the screen and the user event. And then we change the description and say when clicking the delete button instead of when executing a method. So it's more through the eyes of a users, of a real user. Then we initialize a user with this user event method that we get from a testing library package. And next we simulate how a real user would interact with our component by clicking on it. Then again, we use this semantic selector method, find a role and yeah, we click on this button in this case. And that way we've decoupled our component test from implementation details as well.

Okay. So much for decoupling from implementation details, which is important for us to have tests that don't or that enable us to refactor our code instead of being in the way. Because if we couple our tests to implementation details, we not only are they that useful but they even make it impossible for us to refactor our code because they are coupled to the code. And the last way in which we want to decouple our tests from, in this case, the user interface is, yeah, decoupling our tests from the user interface. You might wonder, how can we decouple our tests from the user interface? Or what do I even mean by decoupling the tests from the user interface? So in a perfect world, a perfectly decoupled test would read the same. And it doesn't matter for which platform we are writing the application, or which application for which platform we are testing. So imagine we're having a voice-controlled Alexa. I hope I don't trigger anything here. We have a voice-controlled Alexa application. So a perfectly decoupled test would work for a voice-controlled application the same as for a web application, or for a mobile application, or maybe even for a physical notebook, if we stay in the domain of a shopping list application. So let's take a look at an example. Here we see the test from before again, which now is decoupled from the framework, is decoupled from implementation details, but it is coupled to details of the user interface. Like, in this case, we rely on the fact that we have a URL. And for voice-controlled applications, we don't have a URL. For mobile applications, we don't have a URL.

Only for web applications we have a URL. So this is a user interface implementation detail. And also we rely on the fact that we have input fields. And for voice-controlled applications, we don't have input fields. The same with buttons. So those are all ways in which we rely on the user interface.

And how can we fix this? We can fix this by writing our own domain-specific language. So the domain being the shopping list domain. And a domain-specific language uses words that are common in this domain. Like, here we say, we can open our shopping list. And it doesn't matter if it's an Alexa, voice-controlled application, which you need to somehow trigger. Or if it's a mobile application, or maybe even if it's a physical notebook, you have to open it. Then adding an item. Again, it's the same concept everywhere. You can add an item via voice command. You can add an item by writing it down on a physical notebook. You can add an item by pressing a button, writing something in an input field. Last but not least, we expect that the newly added item appears somehow on the list, or is somehow saved on this list, which again doesn't matter. If we write it that way, it doesn't matter which kind of application we are testing.

So why is this important to do? Why do I think this is useful to do? Not because it's very realistic that we write the same test for multiple platforms. Although, with a system like that, we could even do that. I think that's not the most important part. But the most important part is that, in the same way decoupling from implementation details allows us to refactor our code, decoupling from the user interface allows us to refactor our user interface. So we can really make changes to the user interface. Maybe some day we have a button to trigger some interaction, and the next day we want to have a swipe command or something like that. So details of the user interface can change any time and that's why I think it's useful we have this decoupling. So I have to be fast, I hope I can do it. So here a quick look at how we can implement this. So here we see it's just a wrapper around the methods we saw previously. And yeah, now you might wonder, yeah, but if we have to change, or if we change our user interface, we have to change those lines still.

But the important fact is that you might add items in dozens of tests. And if you change something about the way how you add items, then you only need to change this one line instead of all those dozens of tests.

Okay, so this was everything about decoupling. So decoupling from the test framework. So we can switch test frameworks or choose the best framework for the job, so a very fast framework or a framework with a nice user interface. Then decoupling from implementation details, so we can freely refactor our code. And last we saw decoupling our tests from the user interface so we can change things around the user interface. And we have this nice domain specific language which reads very nicely even for people who don't know about coding, for example. That was about decoupling tests from the user interface.

If you want to learn more about visual tests, so making sure that changes to your code, for example to your CSS, doesn't break how it should look, then I recommend you to watch the talk by Ramona on May 15th about visual testing.

So at the beginning I told you about two important factors, how we can write good tests. One was decoupling from implementation or decoupling in general. And the second important or the first important part I said to you was writing tests first. And you see here I crossed out the word tests because I think it's very useful to think about specifications instead of tests. Because if you think about it, a test is something you do to a finished product. So if a product comes down from the production line, for example, you test it if it works as intended. But specifications, typically you write before producing something. So you specify how it should work before you produce a piece of code or even a physical product.

Why is this important? Writing tests first forces us to think about the public API of our code before we implement it. And usually this leads to better code. Second, we get very fast feedback. If you think about it, you always test your code. Either you do it manually by clicking through your application, for example, or you do it automatically if you have the specification already from the beginning. Then testing the code you just wrote happens automatically in a couple of milliseconds, which is much faster than you can do it yourself manually. And what all of this leads to is that we usually come up with a better design.

Now you might wonder how does this look in practice? How can you write code or how can you write tests before the implementation? So I want to show you. Here we see the test from the end. Basically, it's with the domain-specific language already. Now we want a feature to delete items by clicking the Shopping List items. So it should be possible to remove items, just like that.

And for this we have to simulate that we already have items in our Shopping List and we can do this here with a precondition. So we can see, say, here, has items active, for example. So now there are already items in our Shopping List when we start this test, then we open List, and instead of adding an item, we want to remove an item. So we want to remove the bread item and instead of expecting it to be on the list, we expect item not on list. Okay.

Now we can run those tests and I want to run them in Playwright, classic. Yes. Why doesn't this work? Why doesn't this work? Are they already running? Let's just try again. Okay. Sometimes, it works to just try it again. So here we see the wonderful new Playwright UI. So when Debbie isn't on a conference, I think I have to do her part and show you how amazing Playwright is. And here we can see the first two tests we already had succeed. But the last test we just added doesn't work because the item is still on the list, although it shouldn't be there anymore. Okay.

So now, we could start writing the code, but I want to do more. I want to also add a component test as a stepping stone to fulfill the larger N20 tests. So I already prepared the component test in here because I know that I'm very short on time. It's already curator time. So yeah, deal with it. I want to show you. I'm finishing quickly. So here we have the component test, it's the same as before. So I don't go into much detail right now. But what I want to show you is that I can run them now. And now, it says me exactly what my next step is. Because it tells me that it can't find a button element because there is no button element. So we go to our code. Let's make this big. We go to our shopping list and we see this is a diff.

So of course it doesn't work. Let's change it to a button and save. And now our test tells us, yeah this line now works. So the first step we succeeded. But there is no remove event emitted. So let's change this as well. And say, add click equals. Now we have to define emits. And define emits like that, I hope. There's a typo. Yeah, I have emits equals, define emits like that. Can you spot the error? Ah, thank you.

Okay, now we can say emits and remove. And now we are a step further. Now we succeeded with that one, but we don't emit an ID, so we can change this as well. And say item ID. And now this component has succeeds because our components does what it's supposed to do. It emits an event, but our intent test still does fail because now we also have to listen for this event in our shopping list app component. Here we have the shopping list component we just changed. And here, we can now listen for remove event. And let's copy this add item function and say remove item. We get an item ID here. And we have, by our repository, we have an update method. And this update method takes an item ID, and then we can update the state to be inactive, I think. And then we also update the list. Now, we can use this in here. We can say remove item, save it, go to our end to end test and see it succeeds. And you can also see here, it works. So that way, we can work in a test driven way. Thank you.

So write tests first, decouple all the things and enjoy your work more. And by the way, I'm writing a book about all this stuff. So if you want to check this out, you can go there and see it there.

So, Marcus, thanks a lot for this. I'm not sure if you wanted to show us a bit more. Feel free to join our Q&A session, but at least you run Playwright. That's the best thing when you just have a problem, change nothing, run it again and it works. Yeah, I think that's right.

Okay, so let's start with our session. The first question, which is not a question, let's keep quiet in the back so people can actually focus on the talks. Okay, so just be a bit more quiet. Talk related question, why would I want to change my testing framework? Stubborn developer. Yeah, that's a good question. Usually you probably don't do it that often, but I see it in two ways. First of all, sometimes you do it because now, for example, you see Jest, the development of Jest, for example, slows down, and there are also ... For example, at my work, we are using Nightwatch Jest. Maybe you don't even hear about it because it's not really the newest thing, the trendiest thing anymore, so there are a lot of cases where I think you have or you want to switch test frameworks because although maybe the old one still works, but new developers want to use the new and shiny tools. They want to use Playwright instead of these old tools. So I think it happens quite a lot, and the second way I want to think about it is we also do database extractions, so we don't do SQL queries directly in our code because it's not that often that we actually change our database. It's probably the last thing we change, but it's still a good practice because it also enables us to be more flexible with our code. Totally makes sense. Once I wanted to switch from JESs to VTests and I just didn't do it because everything was coupled too tightly.

The next question. On our webapp, the bugs we had were really hard to cover with tests. We did not get bugs with UI. Our bugs were in frontend data processing. Are these not a good use case for testing, or did we miss a bit? I would suspect that you didn't write tests first because that's a typical sign. That was also a big problem with the application we wrote, that I told you at the beginning, is that our data structure was very complicated. We had a lot of global state, which made our whole application very hard to test. If you have problems covering those bugs with tests, then probably you should try to do more test first, more test-driven development, and then you might see the patterns that make the components hard to test.

And then you might be able to refactor and have more tests that are really useful. Okay, thanks Marcus. The next one is what do you think about using additional data attributes instead of CSS selectors? Yeah, that's one possible way in which how you can decouple your tests from implementation details is by using data QA or data minus test attributes. It's also a good way to do it. It was my favorite way to do it for a long time, but then I stopped because I feel the semantic methods, select methods I showed you make it much more pleasant to read your tests and are even a better way of decoupling the test. But it's absolutely a good practice to use those data attributes as well. Nothing wrong with it. Yeah. Pretty fast to start and pretty simple.