CodeceptJS is BDD-style end-to-end testing framework which plays nicely with all popular testing engines. CodeceptJS has built-in support of PageObjects, DataObjects, plays nicely with TypeScript, and even has its own UI app! In this session, we will take a look how CodeceptJS can be paired with Playwright to make the testing simple and effective.
Play it Right with CodeceptJS: An Introduction to Supercharged End-to-End Testing
AI Generated Video Summary
Concept.js is a BDD style testing framework focused on browser testing, offering easy switching between engines and Cucumber BDD support. The Pulse feature in Concept.js allows for interactive test execution and editing. Concept.js provides a user-friendly interface, reporting options, and supports parallel execution. CodeSap.js offers strategies for dealing with slow-loading elements and provides flexibility in browser control. Contributing to Concept.js is encouraged, and CodeSap.js is a good choice for advanced features and full browser control.
1. Introduction to Concept.js and Testomato
Hello. I hope you enjoyed the conference, and I hope you learned a lot today. Lots of new tools, new approaches. You met wonderful people, and I'd like to jump on the train and introduce you to a new, very cool tool, Concept.js.
So I would call it supercharged end-to-end testing. Well, because I can, and because I really do enjoy testing with Concept.js. So my name is Michael, and I will introduce you into this tool and show you the problem it actually solves. So I am from Kiev, Ukraine. I am actually a web developer, but I also work as a test automation engineers to practice, to see how people, what problems people do have and going with automated testing. And that's how I take this experience to make it an open-source testing frameworks like, surprise, surprise, Concept.js.
2. The Era of Uncertainty and Choosing the Right Tool
Do you remember the days when everything was stable and predictable, and Selenium was the only testing tool for browsers? Now, in the era of uncertainty, we have an ecosystem of tools like Cypress, WebDriver, Playwright, Cucumber, Jest, and Mocha. Choosing the right tool is challenging, especially when clients request specific features like writing tests in Gherkin or supporting Internet Explorer and Safari. We must be prepared to adapt our tests to different environments, including mobile browsers.
So I would start my talk with the question about do you remember that old good days when everything was pretty predictable, everything was stable, and I'm not talking about the things like that you could travel to all the places you want, you could talk to people you want to. I'm talking about just engineering stuff that previously we had only one tool. It was called Selenium, and we used it to test everything that is related to browser. If you need Selenium, you would probably choose Java. If you need, maybe you will need Python or Ruby, but you will choose Selenium. It's a certain decision. There was absolutely no alternatives on these days.
3. Introduction to Concept.js
Concept.js is a BDD style testing framework focused on browser testing. It delegates test running to various engines like Playwright, WebDriver, Protractor, Puppeteer, and Testcafe. It supports easy switching between engines and offers Cucumber BDD support. Concept.js makes tests readable by supporting various ways of clicking elements. It's easy to install, comes with Playwright by default, but can be used without it. Start with a new project, select a browser engine, answer a question, and create your first test.
The previous year we received Playwright from Microsoft. Previously we got Puppeteer, there was TestCafe, Cypress. So lots of new tools are coming out here and there and maybe 2021 brings us something new. Will we be able to switch to it? If it's so new so-called Cypress killer, Playwright killer, absolutely anything killer, maybe we should try it as well and without rewriting tests. Is it ever possible? So this question we can address to concept.js.
So unlike Cypress, concept.js doesn't try tests on its own. It delegates the running to Playwright, to WebDriver, to Protractor, to Puppeteer, to Testcafe. So concept.js can easily switch execution between all those engines. And yes, it comes with Cucumber BDD support in case your client really needs it. So here is the test in concept.js. As you can see, you can read it. Yeah, it's very straightforward. You don't need to look into documentation. You don't need to think of what is happening on the screen. You can already understand everything that is happening here. So concept.js supports clicking by button names, by button tags, by CSS, XPAS, everything. So you can make your test as readable as possible when writing tests with concept.js.
So concept.js is pretty easy to install. It's just one command. And by default, it comes with PlayWrite because I think PlayWrite is really awesome nowadays. And I would choose PlayWrite for my next test because it's multi-browser, multiplatform, and I would recommend it to everyone. But concept.js can be installed without PlayWrite. It's just an opinionated decision which you can easily switch. So concept.js comes with a demo project, but let's start with a new empty project. You will need to initialize your tests and select browser engine. Select browser, answer this boring question, and create a first test. So when we did this, we get into the first test created.
4. Introduction to Pulse Feature in Concept.js
The test should contain only these two steps: open a page and launch the pulse. Pulse replaces all your tests that need to go after they open the page. Let me show you. I'm executing this test in my browser. It opens Airbnb website with the playwright. The execution is paused, so I can open my dev tools and think of what to do next. I can navigate through the page, pick different elements, and write commands in real time. After finishing the tests, I go back to the editor, copy the commands, remove the pause, and voila, the test is ready.
The test should contain only these two steps, open a page and launch the pulse. Now, I'd like you to introduce another cool feature of concept.js is pulse. Pulse replaces all your tests that need to go after they open the page. So pulse is awesome.
Let me show you. So I'm executing this test. It's running in my browser. It opens Airbnb website with the playwright. And after the page is opened, the execution is paused, so now I can open my dev tools and think of what should I do next. You see this is my terminal. You see this pulse message here. So when I'm not sure what should I test, what should I start with, I'm just starting with a pulse. The page is opened as the dev tools are opened. I can navigate through the page and pick different elements, pick different commands, try them, and this is how it goes.
For instance, let me show you how we deal with calendars here. I have my dev tools opened on the bottom and terminal on the right. I write my commands into the terminal and I check if I can find elements by the locators I just picked up. For instance, I found that I can select date with Test ID Locator and I try it. Yeah, it worked and the date was selected. Then I press key Enter and I see that browser acts correspondingly. So I continue to do my actions by writing commands, selecting elements, and this is like I am writing my tests in the real time.
So when we finish picking up comments and we finished our tests, we go back to the editor and our comments were saved into CLI history file. What I do, I copy my commands from this file, only successful comments were saved. I remove the pause and voila, here is my test. It's included all the comments I had. So I was executing this browser once and I didn't do any blind guesses, blind searches. I executed the test and the test is here. By the way, I could write my comments and I have very nice auto completion support. So everyone can start concept GS because the comments are very easy to catch up. For instance, this is the test we finished with.
5. Switching Engines and Reporting in Concept.js
Junior engineers can easily read and understand Concept.js without needing to learn APIs of different frameworks. Switching browser engines is simple by changing the configuration. Concept.js provides built-in UI and graphical interactive test runners, as well as reporting support with options like Concept.UI, allure reports, and Report Portal. Reports are crucial for understanding test results and Concept.js offers various options depending on budget and needs. All tools mentioned are open source.
But as you see, it's very easy to reach so junior engineers can read it and they don't really care of how it is executed. They don't need to learn APIs of different frameworks, only these commons, which are very well documented and auto-completed.
How hard is to switch browser engine? Basically like this, we go to config and we change from player to web driver and we can expect that the test will be executed similarly, not the same, where because you see the web driver will work a bit differently than play, right? Maybe some points you will see crashes, but the idea is that the commons are the same, so all the difference can be easily fixed with maybe add some weighting, change some locator. So for most of the tests, the transition will work smoothly for 95% of test code. You will require no change.
So this solves the problem I mentioned. So if you need to transform your test to play, right, you can easily transform. If you need web driver, you can just change config. Maybe you will need to change to test coffee for internet explorer support, this also works. And yeah, this is really powerful.
Concept.js also comes with built in UI and graphical interactive test runners that can show step-by-step report of all executed tests. So you can use it as a reporting tool. You can see how the execution goes. And it has also some features that Terminal has, but some more advanced as well. For instance, you can keep one browser without relaunching the test and switch easily between headless and window mode.
Also in Concept.js, we have built in reporting support. So one of the best supported tool here is Concept.UI for sure, because Concept.UIs are done to Concept.js. But the screenshot on the right is not from Concept.UI. Maybe you already guessed what it is. It's allure reporting. So Concept.js has built in support for allure reports. In case you need better reports to show to your clients or just to see what was happened on CI in production so you could track the changes on your tests. We also do support the more enterprising technologies like Report Portal. We have official Concept.js agent in Report Portal organization. So you can have Report Portal reports with Concept.js and Testomatio, the tool I'm working on is also tries to bring better reporting to your end-to-end tests. So reports are pretty important in nowadays testing because everything can be shown from a console and for sure your managers, your clients won't look to console to see what tests are passed, what tests not, for what reasons. They want a clean, nice graphical UI to understand this. This is why Concept.js provides so many option for reports because depending on your budget, depending on your conditions, you can choose whatever tool you want. By the way, all those three tools are absolutely open source. You can start working with them and with Concept.js today.
6. Important Features of Concept.js
Concept.js supports parallel execution by workers, provides a solution for failed steps, and takes the traditional concept of page objects. It also offers Cucumber BDD support and the ability to communicate with your application via API. Concept.js focuses on end-to-end testing and is free and open source.
So let me sum up some important features of Concept.js. Not mention features of Concept.js because you already heard of switching between engines, interactive pause, easy to learn syntax. What else do we have? And yes, I was already showing this a bit. Maybe you have already seen it.
So we support parallel execution by workers. So very modern, very elegant API. You can write your own custom runners so you could parallelize your tests however you want. For instance, group run tests in two groups, in two browsers from Firefox or make four groups at Safari. So you can split tests however you want into groups, bring as many browsers you need and have parallel executions and have single report after this. That's really cool.
Also built in, and it's a simply working solution for failed steps. So automatically tries for steps. So if some element is not found on page on first try, you won't even notice it. Concept.js will retries this and wait for it to appear without even notifying you. So it works for any command, click, see, so whatever it is. So it just works, just don't touch it and you're fine.
So another cool feature I really like is if you have API and you could communicate with your application via that API, for instance, you could create a fixtures through this API, delete fixtures. And it is much easier to do it through the REST API interface instead of web interface. So you can easily combine them together. And that's my final points. Concept.js is a testing framework and what's different from other, it focuses on end-to-end testing without executing your end-to-end test by itself. So it tries to take the best user experience into itself to make testing as easy as possible. So Concept.js is absolutely free, open source.
7. Contributing to Concept.js and CodeSapJS Features
You can contribute to Concept.js. Different engines serve different purposes, allowing you to choose the right tool. Cypress is popular for its developer-friendly approach, while WebDriver is suitable for QA engineers who require more browser control. We have received questions about CodeSapJS's multi-browser mode compatibility, and it does support it. CodeSap delegates certain trade-offs to the underlying engines, making it more flexible than Cypress. If you need full browser control and advanced features, CodeSap js is a better choice.
You can contribute on it. There are already big and small companies using Concept.js and joining it. So join our team, join our community, try Concept.js on your own to check it and it works really great with Playwright. Thanks for watching, my name is Michael and see you soon.
Yeah, I thought that Cypress will take even greater numbers here. I'm pretty impressed that someone is using Concept.js. What I wanted to show with this polls that there are different engines and for different purposes, you can choose and the tools that fits right for you. So I think the Cypress is very popular because it's very developer friendly and it took on board lots of developers who didn't like testing, who didn't want to do this and browser testing seemed to them complicated because it was always a product of Selenium to deal with. And the part is WebDriver is more for QA engineers who work on end-to-end testing because they need more tool browser control. So I think it's like two audiences here and yeah, they're quite different but we need to engage all of them to collaborate and make better tests.
That makes a lot of sense. We've also started getting questions for your talk. So I've got a few questions for you, if that's okay. I'm gonna start with them and I'm not gonna mention the names of the people who have asked the questions because I'm gonna butcher the name so there's no point. But one of the first questions was, does CodeSapJS work in the multi-browser mode? For example, a pair of browsers are interacting with each other.
Sure, sure. We had this feature implemented like year ago, no two years ago. So yeah, it's quite some time. So it was working, it was better tested and yes you can use it. Or two different browsers if you want it.
Okay, okay. So that's really good. Somebody made the comment about CodeSap having different trade-offs like Cypress does. Cypress for example, says it doesn't have much tab support and you can't use it to drive two browsers at the same time. How's CodeSap fared compared to those?
CodeSap doesn't solve these problems. It delegates it to the engines that already solve this problem. So this is why I prefer tools like Playwright or Puppeteer which have full browser control, and this is why I dislike Cypress for most of my testing work. But yeah, this is user preference and whatever better fits to your job. So if you need to test single-page application with the single components, Cypress is really fine. But if you need full browser control and more and more features, CodeSap js is better because you can choose the engine while you write your tests.
8. CodeSap.js Strategies and Limitations
CodeSap.js offers strategies to deal with slow-loading elements, including explicit waits, manual retries, and retrying the full scenario. There are no restrictions in CodeSap.js itself, as it delegates browser control to Playwright. However, if you need special APIs, you can write Playwright, Puppeteer, or WebDriver code. Recording interactions in a browser is not recommended, as the interactive pause mode in CodeSap.js provides more flexibility and effectiveness in testing.
So the code base is growing, but you're still able to choose a better engine if you need it. That makes a lot of sense and I like that, especially since people were really skewing towards one of the engines out there.
How does CodeSap deal with things that take a while to appear in the page? For example, a single-page app? Sorry, how does it, what? How does CodeSap deal with things that take a while to appear? So they're slow to load on the page because it's a big page, like a single-page application where everything kinda has to load before you can interact with it.
Yeah, so the problem with single-page application, we don't know is it loaded or not. So it never says us anything about it. So there is no state of this application. So we have like different strategies on this. The default strategy is outer, trying each step. So each step in the test will be automatically retried on failure. We have explicit waits. So you can wait for anything you want there. You can add manual retries. And if you have very, very bad applications that not very flaky, you can retry the full scenario. This is how it also works. I think that's all strategies, but maybe there are a few more of them. But this is that I use regularly. And that works.
People want to know more about the restrictions of using CodeSapp.js. Are there any restrictions like file uploading or working with iframe? What are some of the limitations of the framework? That's quite incorrect questions because I really feel it like Cypress auditory and you can't do it because CodeSapp.js is not driving the browser. It's delegated to Playwright and Playwright has no restrictions. So the only restriction you need to understand that CodeSapp.js can't do everything that Playwright can. So you need to write some Playwright code or Puppeteer code or web driver code if you need some very special APIs to be used. But yeah, you can do everything that you can do with Playwright, Puppeteer, web driver, et cetera, with CodeSapp.js.
Okay, that makes a lot of sense. That makes a lot of sense. Someone wants to know if you can generate CodeSapp.js test by recording interactions in a browser? I'm very skeptical about this idea. So I tried to use it, and I switched to this interactive pause mode. And I found this much more effective because I'm not just recording my actions, I'm testing it in real time, which I can't do with a recorder. So this is much more flexible and effective as for me as a tester. That's really good.
9. CodeSapp Auto-Reply Feature and Q&A
When CodeSapp does the auto-reply feature, is that recording only on success on the first try, or for example, on the third try? The person asking the question has a concern about retries that can mask intermittent bugs. I would like to talk about it after my talking in the room. Please join Michael for more questions. Thank you, Michael, for a talk and for joining me for the Q&A. Thank you for inviting me. Thank you for the opportunity to be here with you.
There's a lot of questions coming in from Michael. We won't get to all of them. We'll take one more. So when CodeSapp does the auto-reply feature, is that recording only on success on the first try, or for example, on the third try? The person asking the question has a concern about retries that can mask intermittent bugs. So how does the auto-retry record happen? I think it's quite a technical question. I'm not sure I will fit in time to answer it, but I would like to talk about it after my talking in the room.
Oh, perfect. In that case, that's a perfect like for me to remind everybody that they can join you in your speakers room on spatial.chat. The link to join is on the live stream timeline, and please join Michael for more questions. We won't get time to go into any more. Thank you, Michael, for a talk and for joining me for the Q&A. It was really good. Thank you for inviting me. Thank you for opportunity to be here with you.