Test Effective Development

Hello everyone and welcome to the Test Effective Development Introduction Talk. I'm Shai Resnick and today I'm going to teach you a new way of thinking about testing that will change your testing life forever and will boost your productivity. So let's begin. As I said, I'm Shai Resnick. I'm known as the Test Effective Guy and sometimes also known as the angular testing Guy. I'm also a Google Developer Expert and I founded Hirers.io, which is an education and training company that teaches developers how to write more cost-effective tests in a fun and entertaining way. Also, my talks have been viewed by over 170,000 developers and a few random people, I'm guessing. Hey, double-hi here. Okay. So my main goal in life is to help busy companies become more cost-effective via testing transformation. That means that we take your busy company with your schedule, not stop in development, and find a way to integrate testing into your day-to-day. If you're interested in that, let me know. Okay, so we could take your developers from sleeping like this into sleeping like this, or as I like to say, test right, sleep tight. Okay? And it's not only about sleeping tight at night. It's also about becoming more professional so you could make the changes to your code or apply this new technique that you learned without the fear of introducing new bugs into production. And if you want to find out your biggest testing mistakes, I prepared a quiz, a few questions that you can answer, and we'll show you what your mistakes and how to fix them, and we'll share with you some cool new free resources about testing. So check out in this link or in this QR code, it'll take you to the same link and get to that quiz. Okay, now a disclaimer. So I'm known, if you search my name on YouTube, you might find crazy talks of mine. It's like, you know, sometimes it's a stage play and sometimes it's a game show or a rap song or a magic show about tests, or one of my personal favorites. This is a talk I did in 2018 where I hooked a device that can read minds and I share with the audience what my thought process and all the shenanigans that go through my head while I'm test driving an app. So even if you're not an angular developer, you'll find value in this lecture. So check it out. But so today I'm not going to do this crazy talk because I have a weird situation here. We have a problem with our Wi-Fi. We are sharing the Wi-Fi signal with all of the other neighbors here in the office building. And there's a weird phenomenon today that, you know, people are going over other people's transmissions. So you might see glitches or you might see what other neighbors of mine here are watching. So I'm sorry about that. I apologize in advance for anything you might see. Okay. I didn't want to, that's why I didn't want to create like a crazy talk today because I don't know when these transmission interruptions will happen. So I'm just sticking to the old fashioned educational talk and that's it. So this is the disclaimer. Sorry about that. And let's hope, let's hope we have a smooth process. Okay, cool. So now let's begin. Okay. So back in 2009, I was a developer in my day job and I had a side project at night. So I worked from nine to five on the day job and until 2am on the startup and that's where my days. And I was in this, what I call the circle of doom because I was rushing and trying to meet all the deadlines and moving forward fast. I produced poor code quality that produced unexpected bugs that in turn produced more poor code quality because of the fear of the unexpected bugs. I got like so scared. Okay. And I didn't want to change and improve the code. And that led to frustration and trying to destroy the computer or I don't know. And that led even to a more scary place where I was in a feeling of a quicksand where every time I fix a bug, I introduced four more bugs into, into the app. And I, it felt like I'm drowning more and more into the quicksand. And I started having this dizziness and, and I couldn't breathe and this attacks and I didn't know what's wrong with me. So I went, I actually went to the hospital and got like checks when it became more severe. And they actually told me that I have nothing wrong with me physically. And they suggested that I might have panic attacks. And I did, this is what I had panic attacks from not meeting the deadline. And you know, I missing the deadlines and I have more bugs to fix and don't know. I have this uncertainty on when more bugs will pop up. And I, it was truly a scary place in my life. So then I decided that I need to work smarter, not harder. Because I thought, okay, what do I really want in my professional life? I want to do meaningful work. I want to upgrade my skills and I want to live a balanced life. So I don't want to, you know, miss holidays or, you know, birthdays or family with friends back then, my girlfriend and what prevented me from doing all that were unexpected bugs. So I then decided I'm going to learn how to fix them and talk with friends and suggested to write tests. So I just learning about testing and I met this like weird terms like unit and integration and then to end and didn't know what to choose when. And it was... Sorry, like I told you, okay, let's forget about what we just saw. Okay. So I had all these terms like unit and integration and didn't know what, you know, what to choose and what to write and how to write them. And somewhere I read that unit tests are for developers and integration and end to end are for QA. So I decided to start with unit tests. So then at the start, it was like very difficult to understand what am I doing. But after a while I became better at it and better. And I thought to myself, okay, I got this. But then, you know, I started having bugs, like 100% coverage or 9% coverage, but still had bugs in production. And then I realized that unit tests are not enough. I also need integration tests. So I tried. I really tried. So I tried to write integration tests and then start again. It was very difficult, but then in the end it was still difficult because integration tests are super difficult. They look like easy in the start, but when you realize you have so much logic and moving pieces in your code and all of this like psychomatic complexity and other terms that makes you sound smarter than you are, but basically saying that you have lots of code path in your app, you know, decision points and stuff like that, that made integration tests much harder and slower to run. And that led me to write only tests for the happy path when stuff are successful and not to check error situations, stuff like that. And that led me to have less confidence in my integration tests because I realized that it's just like an illusion. They're not really give me full confidence in my code. And I always kept feeling that this blanket is too short. You know, every time I try to use a new technique, you know, integration or unit or something like that, I always like suffer from something else. So after a few years of testing, I became a testing consultant and help companies with their code. And I taught them the same traditional ways of like the testing pyramid and all that stuff. And I started to get frustrated because I didn't have the results that I wanted to have. And this is me frustrated. Okay. And I came to the conclusion that I had enough, okay, we need a new approach. We need a better way to do stuff. Okay. So I started experimenting, you know, with strategies. And then I had a few epiphanies and learned the new strategies that worked for me. And once I started to apply them into my client's code and to my own code, I finally got to the place where I can change my code with confidence. And I'm not suffering from all the stuff that I suffered when I used it more traditional ways. And I want to share the epiphany that I had, because again, like I told you, I want to do meaningful work and upgrade my skills and live a balanced life. And what I thought the only thing that stops me from doing it were unexpected bugs. But in fact, it weren't just unexpected bugs, it was also inefficient tests. So I just switching from time wasted on fixing unexpected bugs to time wasted on fixing inefficient tests. So then I thought about, okay, what can I do to handle unexpected bugs? I need to increase my testing confidence. And what can I do to fight inefficient tests? I need to increase my testing efficiency. I know, kept them obvious. But it's important to know, because now that we know these two factors, we can view them and view our tests from these lenses. And if we think about like the strategies, tools and techniques that you are used to apply or use in our tests, okay, now when you think about confidence and efficiency and judge them by these terms, we can see that some strategies lead to more efficiency, but less confidence and some lead to more confidence and less efficiency. For example, end to end test, where you have lots of confidence because all the moving parts are integrated together and you test everything in one place, but they are the least efficient test out of them all because they run the slowest and they require more code and they fail more often and stuff like that. So although you can never reach a 100% perfect point, okay, you always suffer from bugs. The point is to find a balance. The balance to the confidence and efficiency we have from a specific strategy term or tool. So then I thought about, okay, what can define this balance? How can I use one single term to define this balance between the confidence and efficiency? And the only thing that came to mind was cost effective. How cost effective is this strategy? And then I thought it was too generic. So I kept thinking about, okay, cost effective, cost effective. Maybe I can call it test effective. And that's when I came up with test effective development. Okay. And the idea here is to find better terms, better strategies, and better tools to write more cost effective tests. So there's lots of topics to cover with test effective development. But today, because we only have limited time, we're only going to focus on two strategies. Okay, so we'll focus on two strategies, okay? The thing that I learned about being more test effective is if you want to be more effective, you need to be more specific. For example, if you go and want to do a surgery, you want to go to a surgeon who knows how to use the right tool for the job at the right time, and not like a maniac with a chainsaw that just like know how to use this one tool. Okay. And the idea came from, actually from Elon Musk and his first principle thinking method, where he applied it to building reusable rockets by going all the way to the first principles and trying to build back from that better solutions. So that's what I applied. And when I thought about unit integration, I said to myself, okay, let's try to find what are the first principles behind these terms and try to build them back up and maybe come up with new terms. Okay. So when you think about an app, let's say the part of the app like classes or functions or something like that, and the connection between them, like they import each other, this is our app. Okay. And when you think about testing/talks">unit testing, the confusion there is that sometimes you have a unit that describe one class, for example, depending on which book you are reading. And sometimes it defines like two classes together that represent one idea. But then when you learn about integration tests, you learn that it's the same thing basically, it's two pieces together in integration and you test them both. This is also called an integration test. And sometimes, again, depending on which book you're reading, it refers to your code against external code that you don't own. So this is an integration test. So there's a lot of confusion. And then because of this confusion, I decided not to use these terms and to actually dig deeper. And when I dug deeper, I discovered new testing dimensions. We have several testing dimensions that we can analyze. The first one is boundaries. The second is action scope. And the third one is subject type. And we have actually more testing dimensions that I discovered over the years. But today I want to talk about only these first two. I have other lectures and courses and stuff like that where I talk about more dimensions and how to analyze them. But for today's two strategies, I want to talk about these two dimensions. Okay, let's define them. Boundaries. Boundaries are easy. It just means isolated versus integrated. Most of you already refer to unit and integration, and you actually refer to isolated or integrated. And we'll see. So isolated means that we only test one thing. It could be a class, a function or whatever, but it's only one thing. And the rest of the stuff that this thing imports, we fake. Okay, this is an isolated test. In contrast, an integrated test is whenever we test more than one thing together. This is an integrated test. Sometimes it could be the entire thing, which is an end-to-end test. So this is the definition. Action scope means single action or multi-action tests. And what is it? So let's say we have a class with four methods, okay? A single action test will test this one method. And a multi-action test, we test several methods or functions or whatever, two things together in one test. This is a multi-action test. And we can have several combinations of these things, like we can have single action isolated tests or a multi-action integrated test or a single action integrated test. You know, we can have multiple combinations of these dimensions. But choosing the right combination is the key here, okay? You have more effective combinations for different situations, okay? We gently lay our possum down. There. There. I don't know. That's... I'm speechless. Okay, let's just continue. Okay, so let's judge the effectiveness of single versus multi-action tests. In the confidence score, multi-action tests score higher score. It's relative score. It's not scientific. It's relative to each other. Because the more you test more stuff in one test, you get more confidence that you have less side effects. But in terms of efficiency, multi-action tests are much less effective than single action tests. Why? Because of popularity or what I call the influencer curse. Okay, let's see it. Okay, so let's say we have this class here and it has this method. And let's say we have several tests that test this method, okay? Now if this method has a bug, now all these tests will fail. But now let's add another method to the test. So let's say we have all these tests that test this method, but they also run this method as well. Now every time this will have a bug, all tests will fail. And every time this will have a bug, the second method, all tests will fail. Okay? So when you introduce more methods or more actions to your test, you're increasing the probability of failing lots of tests together. And that's the influencer curse. The more followers your code has, like the more tests that follows your code, the less efficient your tests are. Okay? So the conclusion is to prefer single action tests. Okay? There are some times that you want to use multi-action tests, like in end-to-end tests, where the amount of tests are smaller. And I talk about this in my courses and other lectures. But for most tests, you want to prefer single action tests. Okay, now let's talk about the effectiveness of isolated versus integrated. Okay? I've seen a lot of talks about write only integration tests and the testing pyramid or write only unit tests or mostly unit tests or mostly integration and all that stuff. And I followed these suggestions for a while and it led to more confusion on my part because I realized that what's important is the context of your app. And part of this context is the size. Size matters in testing. Okay? So let's define size. On one end, we have the small apps. Example for these can be like microservices or micro-frontends or open source libraries or proof of concept, stuff like that. These are smaller apps. Okay? On the other end, we have medium to big apps, which most of the time, examples of these are monoliths. Okay? If we don't break them into micro-frontends, it's a big monolith with lots of moving parts compared to a mono-repo or something like that. So these are big apps. Okay? These are the terms. Okay? So we have integrated versus isolated on the confidence score. We have a high confidence score for the integrated test and a lower confidence score for the isolated because we lose touch with reality when we fake stuff. But there is an asterisk there on the integrated part because it means that we must cover everything in the integrated and not just the happy path. And most of the time, it's not the situation. That's why it's a caveat. Okay? You know, if in an ideal world where we cover all the possible code path with integrated test, yeah, we have maximum confidence. But that's most of the time isn't the case. On the efficiency score, when we look at integrated versus isolated, integrated is much less efficient than isolated test. And there is a lot of reasons as to why that is. Okay? I suggest watching a great lecture about the scam of integrated test or integrated test or a scam by JB Reinsberger where he covers that in depth. And I have other lectures about where I go deeper into why this is the case. Okay? What's the scam of integrated test? But they are much less efficient than isolated test. And for that reason, what I came to realize is that for small to mid-apps, the more test effective solution is to use a combination between single action and integrated test because that way we don't lose the confidence and we don't lose that much efficiency because we don't have a lot of tests in the first place because this is a small size app. But for big monolith or mid to large scale apps, the more test effective solution there or strategy there is to use a combination of single action and isolated test because that way we preserve a good balance between the confidence and efficiency. I'm just going to ignore that. Okay? Okay. And because I didn't want to keep saying like single action integrated test or single action isolated test and stuff like that, I want to find better terms that encapsulate what I mean. I want to introduce to you now new test types based on size. So type number one, okay, for small apps, I want something that could cover all parts of the app. And then I thought, okay, what's a good metaphor for, let's say I had all these parts in one room, I want to shed a light on all of these together. And then I thought about like, hey, I can use a flashlight. Flashlight, you know, with the ray of light can hit all the parts together. Okay? And then I thought about the name flashlight test, which are short distance. Okay. Think about the small room, single action integrated tests. And these are flashlight test. And this is type number one. Now the other new test type that I thought about was for the big apps. Let's say I had like, it's a big stadium with lots of distant parts. Okay. Sometimes I want to reach this part or this part. What can I use to reach it? I cannot use flashlight because the ray of light can reach just a short distance. I need another device. And then I thought about, I can use a pointer laser. And that way I can reach other like more distant and just those parts because they are isolated. Okay. So then is when I thought about the name laser test. Okay. Which are long distance. Think about a big stadium, single action isolated tests. Okay. So again, to recap, we have flashlight test for small apps and laser tests for big apps. Okay. So you might ask yourself, okay, how do you know when to switch between two strategies? The rule of thumb for me is when the test reach more than 500 lines of code, then I know that probably my integrated tests are becoming less efficient. And then I consider switching to a laser test strategy. Okay. So that's our time for today. Let's summarize what we learned. We learned about test effective development. We learned about the different testing dimensions we have, and we learned about the laser test and flashlight test strategies. Okay. If you want to learn about more test effective strategies, again, this is the quiz I prepared for you. This is the link. This is the QR code. Check it out and you'll get more free resources to learn further about this topic, about test effective development. And with that, I wanted to thank you all for participating in my first ever TED talk. Okay. This is my first. And thank you very much. These are all the details. Remember, check out the quiz and become more test effective developer. Okay. Enjoy the rest of the event. Bye bye. Hello. Hi, Shai. How are you? I was so excited for the talk and I gave my all and I lost my voice. So you can hear my sexy voice in the Q&A now. It was great. It was super funny. I love the videos in the middle. So we had the poll before. So let's take a look at the results. And in the results, I actually didn't knew the answer. So my vote was on the hardware testing and afterwards I was reading about it. So what's your takeaway in regards to the results of the poll? So I'm seeing the results now and it looks like we have a tie between plumbing and hardware testing. And the real answer is hand teasing. Nobody picked hand teasing. It's like 0% or something like that. But no, actually, when you, you know, I always like thought it was regarding to plumbing and then because, you know, they blow smoke through the pipes to see if you don't have any, like, you know, leaks. But it actually comes also from hardware testing. Okay, so hardware testing, the test there was if you plug it in and it doesn't blow up in smoke, it works like you pass the smoke test. So it actually is a combination between so the right answer is both A and B are correct. Exactly. Yeah, that's what I researched afterwards. And the plumbing one came first and then the hardware testing. So that's great. We have a few questions on Discord. So I'll read them to you. And the first one is from Lias and the T-shirt in the question is, can we get a T-shirt with the test rights lip tights? I actually plan. Thank you very much. Yeah, I want to have one as well. I will. Yeah, I will. I'm planning on that. I have a design and I plan to issue like I just needed to find the time to actually see where can I, you know, produce it internationally and stuff like that. But if you'd like to be on the on the list, so hit me up at shai at highres.io and I will put you in the notification list when it will go live. But yeah, I'm planning like a whole line of testing related funny T-shirts. You have created your own terms so you can have lots of different shirts with different sayings. Awesome. Yeah, I'm planning on doing a laser test and stuff like that. Smoke smoke and lasers and stuff like that. So we have another question here, which is, does the size of the app matter when choosing the task, a testing strategy? I like I think it's I answered it in the in the talk. In the end. Yeah. Yeah. So, yeah, this this is the the the whole point that I tried to make was that when we read about the blog post that like suggest a specific strategy, we you know, we used to read it from like, OK, our point of view and we might take like the wrong strategy to our app if we don't apply the or wait for a second and think, what is the size? What is the size of my app? And yeah, does this strategy like does this technique really is the most cost effective technique? And that's why I said that if you judge it all, that's what I try to do. If you judge everything and you have more more aspects and more dimensions to judge. Like I said, like you have the you have a subject and you have all these things that you need to consider. That's what I try to do in my lab here in my office. By the way, this is my family. So what I try to do is to figure out what is the most, you know, cost effective per context. So for context of smaller apps is try to use a combination of single action, single actions anyway. OK, for your like flashlight micro test. Yeah. And do the flashlight test, which is the single action and integrated test. That's how you get the most gains and most efficiency and most confidence. But for large apps, which most of us work in companies where we have still like monolith and big monsters of apps, it's pretty hard to write integrated tests in such an environment in a very, I don't know, repetitive or like a conventional way where everybody know the borders, the right like. Where the test will be very deterministic, right? Exactly. So that's why I recommend, OK, stick to isolated tests for that strategy and use the integrated test for stuff like smoke tests, which I talk about on other lectures. Oh, we have a hand teasing. Somebody answered hand teasing, which is amazing. Thank you. You won. I like that you mentioned and it's not just you, many of the speakers here in the conference mentioned that we should choose the right testing that are more cost effective for our needs, for our context. And I think that's what makes more sense. There's no one strategy that works for everyone. We need to understand exactly where we are, where we are in and where we want to be. Right. Yeah. And that's something that I think we should talk about in the testing world, more like try to go to first principles and try to assess stuff, not by the terms that we're used to talk about, but for what is more correct to my situation. And let's define what is my situation. Am I working in a small app, in a big app? And then, okay, what are the different dimensions? What do I really need here? Do I really like in angular, for example, you know, in react, it's pretty obvious to write component or DOM or test for the DOM. It's more like, you know, it's easier, it's faster. In angular, it's actually faster to write only a test for the classes and not for the DOM. So, again, leave the DOM test for the reusable components, but that's not the common wisdom, let's say, you know, the common thing you will read about. But I personally don't care about what's common. I care about what works in real life situation. So that's what I think we need more critical thinking on these terms. Just a final thought that I have as well is that the same project might start with a strategy and when it evolves, it will have to change its strategy, too, because the app grows and it becomes more complex. So, we have to have that in mind, for sure. It was a great talk, Chao Chai. I really appreciated it. Thank you very much for that. Thank you, too. Thanks. Thanks for having me.