Play it Right with CodeceptJS: An Introduction to Supercharged End-to-End Testing

Hello, I hope you enjoyed the conference and I hope you learned a lot today. Lots of new tools, new approaches, you met wonderful people and I'd like to jump on this train and introduce you to a new very cool tool, Concept.js. So I would call it supercharged end-to-end testing, well, because I can and because I really do enjoy testing with Concept.js. So my name is Michael and I will introduce you into this tool and show you the problem it actually solves. So I'm from Kiev, Ukraine, I'm actually a web developer, but I also work as a test automation engineer to practice, to see how people, what problems people do have when going with automated testing. And that's how I take this experience to make it an open source testing framework like, surprise, surprise, Concept.js. So yes, I am lead developer of Concept.js as well, but also I like do testing with Concept.js. Well, I think it's fair enough to say this. I started my journey into open source testing frameworks with my previous project, Conception, which is very popular in PHP community. And Concept.js takes this experience and brings this to a new level and gets into JavaScript ecosystem. Nowadays, I'm working for my own company and we are building a test management system for automated test, Testomato. Testomato is focused on JavaScript tests. So if you have a total mess with your tests, Testomato can organize it for you. That's the idea. So I would start my talk with the question about, do you remember that old good days when everything was pretty predictable, everything was stable? And I'm not talking about the things like that you could travel to all the places you want, you could talk to people you want to. I'm talking about just engineering stuff that previously we had only one tool. It was called Selenium and we used it to test everything that is related to browser. If you need Selenium, you would probably choose Java. If you need, maybe you will need Python or Ruby, but you will choose Selenium. It's certain decision. There was absolutely no alternatives on those days. But today we live in the era of uncertainty. The JavaScript spawned and it created a wonderful ecosystem of different tools. And today choosing a right tool for the job is a tricky question because we have Cypress, we have WebDriver, Playwright, Cucumber, Jest, Mocha, and wow, how much of things you need to learn today to choose the right tool for the job. We definitely live in the era of uncertainty. And much more stress brings the questions I'm trying to list when I'm asking a client for a new test automation project. Because we can start writing tests, but then client came out and said, hey, hey, hey, I learned about the BDD yesterday. It's so awesome. I want to write this test myself in Gherkin language. Can our tool write tests in Gherkin? Good question, especially if we already have our tests not in Gherkin but in JavaScript. Can we easily introduce BDD level on top of it? Oh, the client came. I need Internet Explorer. You will hate me for this, but we need Internet Explorer because I have lots of enterprise clients. Can your tool easily switch to Enterprise Explorer on the fly when all the tests, especially with BDD and all the stuff were already written? That's a good question. A client came again. I need Safari support. Well, Safari is fair enough. Safari is a modern browser. Everyone uses it. And if our tool can do Safari or not, this also can be tricky for some kind of tools. But we need to be prepared for all those changes all this time. Can we adjust our tests to be executed on all those different environments? And especially for mobile browsers because the world goes mobile and testing in mobile is also a hard requirement that the client may not be fully aware of at the beginning of the journey, but will be asking when you have some tests and they will ask you, could you execute this test on mobile and this test? So mobile browser can be a new requirement as well. And again, it's an era of uncertainty. We don't know what a new tool will come up this year because every year some good test automation tool emerges. The previous year we received Playwright from Microsoft. Previously, we got Puppeteer. There was Test Cafe Cypress. So lots of new tools are coming out here and there, and maybe 2024 brings us something new. Will we be able to switch to it if it's so new, so-called Cypress killer, Playwright killer, absolutely anything killer? Maybe we should try it as well and without rewriting tests. Is it ever possible? So this question we can address to concept.js. Concept.js is BDD style testing framework. BDD style means that we write our tests in human-like language and we try to avoid some complex JavaScript interaction and keeping the code as readable as possible. And it's solely focused on browser testing that makes it different from Mocha or Jest, but it has API for all browser engines. So unlike Cypress, concept.js doesn't try tests on its own. It delegates the running to Playwright, to WebDriver, to Protractor, to Puppeteer, to Test Cafe. So concept.js can easily switch execution between all those engines. And yes, it comes with Cucumber BDD support in case your client really needs it. So here is the test in concept.js. As you can see, you can read it. It's very straightforward. You don't need to look into documentation. You don't need to think of what is happening on the screen. You can already understand everything that is happening here. So concept.js supports clicking by button names, by button text, by CSS, everything. So you can make your test as readable as possible when writing tests with concept.js. So concept.js is pretty easy to install. It's just one command. And by default, it comes with Playwright because I think Playwright is really awesome nowadays. And I would choose Playwright for my next test because it's multi-browser, multi-platform, and I would recommend it to everyone. But concept.js can be installed without Playwright. It's just an opinionated decision which you can easily switch. So concept.js comes with a demo project, but let's start with a new empty project. You will need to initialize your new test and select browser engine, select browser, answer this boring question, and create a first test. So when we did this, we get into the first test created. The test should contain only these two steps, open a page and launch the pause. Now I'd like you to introduce another cool feature of concept.js is pause. Pause replaces all your tests that need to go after they open the page. So pause is awesome. Let me show you. So I am executing this test. It's running in my browser. It opens Airbnb website with Playwright. And after the page is opened, the execution is paused. So now I can open my dev tools and think of what should I do next. You see this is my terminal. You see this pause message here. So when I'm not sure what should I test, what should I start with, I'm just starting with a pause. The page is opened, the dev tools are opened. I can navigate through the page and pick different elements, pick different comments, try them. And this is how it goes. For instance, let me show you how we deal with calendars here. I have my dev tools opened on the bottom and terminal on the right. I write my comments into the terminal and I check if I can find elements by the locators I just picked up. For instance, I found that I can select date with test ID locator and I try it. Yeah, it worked. And this is the date was selected. Then I press key enter and I see that browser acts correspondingly. So I continue to do my actions by writing comments, selecting elements. And this is like I am writing my tests in the real time. So when we finish picking up comments and we finished our tests, we go back to the editor and our comments were saved into CLI history file. What I do, I copy my comments from this file. Only successful comments were saved. I remove the pause and voila, here is my test. It's included all the comments I had. So I was executing this browser once and I didn't do any blind guesses, blind searches. I executed the test and the test is here. By the way, I could write my comments and I have very nice auto-completion support. So everyone can start with Cutsape.js because the comments are very easy to catch up. For instance, this is the test we finished with, but as you see, it's very easy to read. So junior engineers can read it and they don't really care of how it is executed. They don't need to learn APIs of different frameworks, only these comments, which are very well documented and auto-completed. How hard is it to switch browser engine? Basically like this, we go to config and we change from player to web driver and we can expect that the test will be executed similarly. Not the same, because you see the web driver will work a bit differently than playwright. Maybe at some points you will see crashes, but the idea is that the comments are the same. So all the difference can be easily fixed with maybe add some weighting, change some locator. So for most of the tests, the transition will work smoothly. For 95% of test code, you will require no change. So this solves the problem I mentioned. So if you need to transform your test to playwright, you can easily transform. If you need web driver, you can just change config. Maybe you will need to change to TestCafe for Internet Explorer support. This also works. And yeah, this is really powerful. Concept.js also comes with built-in UI, a graphical interactive test runner that can show step-by-step report of all executed tests. So you can use it as a reporting tool. You can see how the execution goes. And it has also some features that terminal has, but some more advanced as well. For instance, you can keep one browser without relaunching a test and switch easily between headless and window mode. Also in Concept.js, we have a built-in reporting support. So one of the best supported tool here is Concept.UI for sure, because Concept.UIs are done to Concept.js, but the screenshot on the right is not from Concept.UI, maybe you already guessed what it is. It's allure reporting. So Concept.js has built-in support for allure reports. In case you need better reports to show to your clients or just to see what was happened on CI in production, so you could track the changes on your tests. We also do support the more enterprise technologies like Report Portal. We have official Concept.js agent in Report Portal organization, so you can have Report Portal reports with Concept.js. And Testumatio, the tool I'm working on, it also tries to bring better reporting to your end-to-end tests. So reports are pretty important in nowadays testing because everything can be shown from console and for sure your managers, your clients won't look to console to see what tests are correct, what tests are not, for what reasons, they want a clean, nice graphical UI to understand this. This is why Concept.js provides so many options for reports because depending on your budget, depending on your conditions, you can choose whatever tool you want. By the way, all those three tools are absolutely open source. You can start working with them and with Concept.js today. So let me sum up some important features of Concept.js. Not mention features of Concept.js because you already heard of switching between engines, interactive polls, easy to learn syntax. What else do we have? And yes, I was already showing this a bit, maybe you have already seen it. So we support parallel execution by workers. So very modern, very elegant API. You can write your own custom runner so you could parallelize your tests however you want. For instance, group run tests in two groups, in two browsers from Firefox or make four groups at Safari. So you can split tests however you want into groups, bring as many browsers you need and have parallel executions and have single report after this. That's really cool. Also built in and simply working solution for failed steps. So automatically tries four steps. So if some element is not found on page on first try, you won't even notice it. Concept.js will re-try this and wait for it to appear without even notifying you. So it works for any command, click, see, so whatever it is. So it just works. Just don't touch it and you are fine. Concept.js takes the traditional concept of page objects and it's built in into the framework so you can easily refactor your code to make it easy to be used in other tests. So page objects, step objects, data objects are all built in. And as I mentioned previously, we have Cucumber BDD support built on top of Concept.js. So this is really awesome because you can combine true BDD with specifications in JavaScript like Concept.js tests. This is important because not all tests should be written in the truly BDD manner because not all tests are features. So because some you need to have regression tests for bugs. So my point is that you should combine BDD approach with classical approach and only Concept.js testing framework allows you to do this. So another cool feature I really like is if you have API and you could communicate with your application via that API, for instance, you could create features through this API, delete pictures. And it is much easier to do it through the REST API interface instead of web interface. So you can easily combine them together. And that's my final points. Concept.js is a testing framework and what differs it from others, it focuses on end-to-end testing without executing your end-to-end test by itself. So it tries to take the best user experience into itself to make testing as easy as possible. So Concept.js is absolutely free, open source. You can contribute on it. There are already big and small companies using Concept.js and enjoying it. So join our team, join our community, try Concept.js on your own to check it. And it works really great with Playwright. Thanks for watching. My name is Michael and see you soon. Yeah, I thought that Cypress will take even greater numbers here. I'm pretty impressed that someone is using Concept.js. What I wanted to show with this poll is that there are different engines and for different purposes you can choose the tools that fit right for you. So I think Cypress is very popular because it's very developer friendly and it took on board lots of developers who didn't like testing, who didn't want to do this and browser testing seemed to them complicated because it was always a product of Selenium to deal with. And the part with WebDriver is more for QA engineers who work on end-to-end testing. They need more to browser control. So I think it's like two audiences here. And yeah, they're quite different. But we need to engage all of them to collaborate and make better tests. That makes a lot of sense. We've also started getting questions for your talk. So I've got a few questions for you, if that's okay. I'm going to start with them and I'm not going to mention the names of the people who have asked the questions because I'm going to butcher the names. So there's no point. But one of the first questions was, does CodeSav.js work in the multi-browser mode? For example, a pair of browsers are interacting with each other? Sure, sure. We had this feature implemented like a year ago. No, two years ago. So yeah, it's quite some time. So it was working, it was better tested. And yes, you can use it. Two different browsers if you want it. Okay, okay. That's really good. Somebody made a comment about CodeSav having different trade-offs like Cypress does. Cypress, for example, says it doesn't have multi-tab support and you can't use it to drive two browsers at the same time. How's CodeSav faring compared to those? Cypress doesn't solve this problem. It delegates it to the engines that already solved this problem. So this is why I prefer tools like Playwright or Puppeteer, which have full browser control. This is why I dislike Cypress for most of my testing work. But yeah, this is like a user preference and whatever is better fits to your job. So if you need to test single page application with a single component, Cypress is really fine. But if you need full browser control and more and more features, ConceptJS is better because you can choose the engine while you write your tests. So the code base is growing, but you're still able to choose a better engine if you need it. That makes a lot of sense. And I like that, especially since people were really, really skewing towards one of the engines out there. How does CodeSav deal with things that take a while to appear in the page? For example, a single page app? Sorry, how does it... What? How does CodeSav deal with things that take a while to appear? So they're slow to load on the page because it's a big page. Like a single page application where everything kind of has to load before you can interact with it. Yeah. So the problem with single page application, we don't know is it loaded or not. So it never says us anything about it. So there is no state of this application. So we have different strategies on this. The default strategy is outer a triangle each step. So each step in the test will be automatically retried on failure. We have explicit waits. So you can wait for anything you want there. You can add manual retries. And if you have very, very bad applications that are not very flaky, you can retry the full scenario. This is what also works. I think that's all strategies, but maybe there are a few more of them. But this is what I use regularly and that works. People want to know more about the restrictions of using code, code.js. Are there any restrictions like fire log loading or working with iframe? What are some of the limitations of the framework? That's quite incorrect questions because I really feel it's like Cypress auditory and you can't do it because concept.js is not driving the browser. It's delegated to playwright. And playwright has no restrictions. So the only restriction you need to understand is that concept.js can't do everything that playwright can. So you need to write some playwright code or puppeteer code or web driver code if you need some very special APIs to be used. But yeah, you can do everything that you can do with playwright, puppeteer, web driver, et cetera, with concept.js. Okay. That makes a lot of sense. That makes a lot of sense. Someone wants to know if you can generate a concept.js test by recording interactions in a browser? I'm very skeptical about this idea. So I tried to use it. Then I switched to this interactive pause mode and I found this much more effective because I'm not just recording my actions. I'm testing it in the real time, which I can't do with recorder. So this is much more flexible and effective for me as a tester. That's really good. There's a lot of questions coming in from Michael. We won't get to all of them. We'll take one more. So when CodeSep does the auto retry feature, is that recording only on success on the first try or, for example, on the third try? The person asking the question has a concern about retries that can mask intermittent bugs. So how does the auto retry record happen? I think it's quite a technical question. I'm not sure I will fit in time to answer it. But I would like to talk about it after my talking in the room. Oh, perfect. In that case, that's a perfect plug for me to remind everybody that they can join you in your speaker's room on spatial.chat. The link to join is on the livestream timeline. And please join Michael for more questions. We won't get time to go into any more. Thank you, Michael, for your talk and for joining me for the Q&A. It was really good. Thank you for inviting me and thank you for the opportunity to be here with you. Thanks for watching, everyone.