Play it Right with CodeceptJS: An Introduction to Supercharged End-to-End Testing


CodeceptJS is BDD-style end-to-end testing framework which plays nicely with all popular testing engines. CodeceptJS has built-in support of PageObjects, DataObjects, plays nicely with TypeScript, and even has its own UI app! In this session, we will take a look how CodeceptJS can be paired with Playwright to make the testing simple and effective.

28 min
15 Jun, 2021


Sign in or register to post your comment.

AI Generated Video Summary

Concept.js is a BDD style testing framework focused on browser testing, offering easy switching between engines and Cucumber BDD support. The Pulse feature in Concept.js allows for interactive test execution and editing. Concept.js provides a user-friendly interface, reporting options, and supports parallel execution. CodeSap.js offers strategies for dealing with slow-loading elements and provides flexibility in browser control. Contributing to Concept.js is encouraged, and CodeSap.js is a good choice for advanced features and full browser control.

1. Introduction to Concept.js and Testomato

Short description:

Hello. I will introduce you to Concept.js, a supercharged end-to-end testing tool. I am Michael, a web developer and test automation engineer. I have experience with open-source testing frameworks and have developed Concept.js to solve testing problems. I also work on Testomato, a test management system for JavaScript tests.

Hello. I hope you enjoyed the conference, and I hope you learned a lot today. Lots of new tools, new approaches. You met wonderful people, and I'd like to jump on the train and introduce you to a new, very cool tool, Concept.js.

So I would call it supercharged end-to-end testing. Well, because I can, and because I really do enjoy testing with Concept.js. So my name is Michael, and I will introduce you into this tool and show you the problem it actually solves. So I am from Kiev, Ukraine. I am actually a web developer, but I also work as a test automation engineers to practice, to see how people, what problems people do have and going with automated testing. And that's how I take this experience to make it an open-source testing frameworks like, surprise, surprise, Concept.js.

So yes, I am lead developer of Concept.js as well, but also I like do testing with Concept.js. Well, I think it's fair enough to say this. I started my journey to open source testing frameworks with my previous project, Conception, which is very popular in PHP community. And Concept.js takes this experience and brings this to a new level and gets into JavaScript ecosystem. Nowadays, I'm working for my own company and we are building a test management system for automated test, Testomato. Testomato is focused on JavaScript tests, so if you have a total mess with your tests, Testomato can organize it for you. That's the idea.

2. The Era of Uncertainty and Choosing the Right Tool

Short description:

Do you remember the days when everything was stable and predictable, and Selenium was the only testing tool for browsers? Now, in the era of uncertainty, we have an ecosystem of tools like Cypress, WebDriver, Playwright, Cucumber, Jest, and Mocha. Choosing the right tool is challenging, especially when clients request specific features like writing tests in Gherkin or supporting Internet Explorer and Safari. We must be prepared to adapt our tests to different environments, including mobile browsers.

So I would start my talk with the question about do you remember that old good days when everything was pretty predictable, everything was stable, and I'm not talking about the things like that you could travel to all the places you want, you could talk to people you want to. I'm talking about just engineering stuff that previously we had only one tool. It was called Selenium, and we used it to test everything that is related to browser. If you need Selenium, you would probably choose Java. If you need, maybe you will need Python or Ruby, but you will choose Selenium. It's a certain decision. There was absolutely no alternatives on these days.

But today we live in the era of uncertainty. The JavaScript's found, and it created a wonderful ecosystem of different tools. And today choosing a right tool for the job is a tricky question, because we have Cypress, we have WebDriver, Playwright, Cucumber, Jest, Mocha. And well, how much of things you need to learn today to choose the right tool for the job. We definitely live in the era of uncertainty, and much more stress brings the questions I'm trying to list when I'm asking a client for new test automation project. because we can start writing tests, but then client came out and say, hey, I've learned about the BDD yesterday. It's so awesome. I want to write this test myself in Gherkin language. Can our tool write tests in Gherkin? Hmm, good question. Especially if we already have our tests not in Gherkin, but in JavaScript, can we easily introduce BDD level on top of it? Oh, the client came. I need Internet Explorer. You will hate me for this, but we need Internet Explorer because I have lots of enterprise clients. Can your tool easily switch to Enterprise Explorer on the fly when all the tests, especially with BDD on all the stuff we're already written? That's a good question. Hey, client comes again. I need Safari support. Well, Safari is fair enough. Safari is modern browser. Everyone uses it. And if our tool can do Safari or not, this also can be tricky for some kind of tools. But we need to be prepared for all those changes all this time. Can we adjust our tests to be executed on all those different environments? And especially for mobile browsers because the world goes mobile, and test it in mobile is also a hard requirement that client may not be fully aware of at the beginning of the journey but will be asking when you have some of tests and they will ask you, execute this test on mobile and this test. So mobile browser can be a new requirement as well. And against era of uncertainty, we don't know what a new tool will come up this year because every year, some good test automation tool emerges.

3. Introduction to Concept.js

Short description:

Concept.js is a BDD style testing framework focused on browser testing. It delegates test running to various engines like Playwright, WebDriver, Protractor, Puppeteer, and Testcafe. It supports easy switching between engines and offers Cucumber BDD support. Concept.js makes tests readable by supporting various ways of clicking elements. It's easy to install, comes with Playwright by default, but can be used without it. Start with a new project, select a browser engine, answer a question, and create your first test.

The previous year we received Playwright from Microsoft. Previously we got Puppeteer, there was TestCafe, Cypress. So lots of new tools are coming out here and there and maybe 2021 brings us something new. Will we be able to switch to it? If it's so new so-called Cypress killer, Playwright killer, absolutely anything killer, maybe we should try it as well and without rewriting tests. Is it ever possible? So this question we can address to concept.js.

Concept.js is BDD style testing framework. BDD style means that we write our tests in human-like language and we try to avoid some complex JavaScript interaction and keeping the code as readable as possible. And it's solely focused on browser testing that makes it different from Mocha or Jest but it have API for all browser engines.

So unlike Cypress, concept.js doesn't try tests on its own. It delegates the running to Playwright, to WebDriver, to Protractor, to Puppeteer, to Testcafe. So concept.js can easily switch execution between all those engines. And yes, it comes with Cucumber BDD support in case your client really needs it. So here is the test in concept.js. As you can see, you can read it. Yeah, it's very straightforward. You don't need to look into documentation. You don't need to think of what is happening on the screen. You can already understand everything that is happening here. So concept.js supports clicking by button names, by button tags, by CSS, XPAS, everything. So you can make your test as readable as possible when writing tests with concept.js.

So concept.js is pretty easy to install. It's just one command. And by default, it comes with PlayWrite because I think PlayWrite is really awesome nowadays. And I would choose PlayWrite for my next test because it's multi-browser, multiplatform, and I would recommend it to everyone. But concept.js can be installed without PlayWrite. It's just an opinionated decision which you can easily switch. So concept.js comes with a demo project, but let's start with a new empty project. You will need to initialize your tests and select browser engine. Select browser, answer this boring question, and create a first test. So when we did this, we get into the first test created.

4. Introduction to Pulse Feature in Concept.js

Short description:

The test should contain only these two steps: open a page and launch the pulse. Pulse replaces all your tests that need to go after they open the page. Let me show you. I'm executing this test in my browser. It opens Airbnb website with the playwright. The execution is paused, so I can open my dev tools and think of what to do next. I can navigate through the page, pick different elements, and write commands in real time. After finishing the tests, I go back to the editor, copy the commands, remove the pause, and voila, the test is ready.

The test should contain only these two steps, open a page and launch the pulse. Now, I'd like you to introduce another cool feature of concept.js is pulse. Pulse replaces all your tests that need to go after they open the page. So pulse is awesome.

Let me show you. So I'm executing this test. It's running in my browser. It opens Airbnb website with the playwright. And after the page is opened, the execution is paused, so now I can open my dev tools and think of what should I do next. You see this is my terminal. You see this pulse message here. So when I'm not sure what should I test, what should I start with, I'm just starting with a pulse. The page is opened as the dev tools are opened. I can navigate through the page and pick different elements, pick different commands, try them, and this is how it goes.

For instance, let me show you how we deal with calendars here. I have my dev tools opened on the bottom and terminal on the right. I write my commands into the terminal and I check if I can find elements by the locators I just picked up. For instance, I found that I can select date with Test ID Locator and I try it. Yeah, it worked and the date was selected. Then I press key Enter and I see that browser acts correspondingly. So I continue to do my actions by writing commands, selecting elements, and this is like I am writing my tests in the real time.

So when we finish picking up comments and we finished our tests, we go back to the editor and our comments were saved into CLI history file. What I do, I copy my commands from this file, only successful comments were saved. I remove the pause and voila, here is my test. It's included all the comments I had. So I was executing this browser once and I didn't do any blind guesses, blind searches. I executed the test and the test is here. By the way, I could write my comments and I have very nice auto completion support. So everyone can start concept GS because the comments are very easy to catch up. For instance, this is the test we finished with.

5. Switching Engines and Reporting in Concept.js

Short description:

Junior engineers can easily read and understand Concept.js without needing to learn APIs of different frameworks. Switching browser engines is simple by changing the configuration. Concept.js provides built-in UI and graphical interactive test runners, as well as reporting support with options like Concept.UI, allure reports, and Report Portal. Reports are crucial for understanding test results and Concept.js offers various options depending on budget and needs. All tools mentioned are open source.

But as you see, it's very easy to reach so junior engineers can read it and they don't really care of how it is executed. They don't need to learn APIs of different frameworks, only these commons, which are very well documented and auto-completed.

How hard is to switch browser engine? Basically like this, we go to config and we change from player to web driver and we can expect that the test will be executed similarly, not the same, where because you see the web driver will work a bit differently than play, right? Maybe some points you will see crashes, but the idea is that the commons are the same, so all the difference can be easily fixed with maybe add some weighting, change some locator. So for most of the tests, the transition will work smoothly for 95% of test code. You will require no change.

So this solves the problem I mentioned. So if you need to transform your test to play, right, you can easily transform. If you need web driver, you can just change config. Maybe you will need to change to test coffee for internet explorer support, this also works. And yeah, this is really powerful.

Concept.js also comes with built in UI and graphical interactive test runners that can show step-by-step report of all executed tests. So you can use it as a reporting tool. You can see how the execution goes. And it has also some features that Terminal has, but some more advanced as well. For instance, you can keep one browser without relaunching the test and switch easily between headless and window mode.

Also in Concept.js, we have built in reporting support. So one of the best supported tool here is Concept.UI for sure, because Concept.UIs are done to Concept.js. But the screenshot on the right is not from Concept.UI. Maybe you already guessed what it is. It's allure reporting. So Concept.js has built in support for allure reports. In case you need better reports to show to your clients or just to see what was happened on CI in production so you could track the changes on your tests. We also do support the more enterprising technologies like Report Portal. We have official Concept.js agent in Report Portal organization. So you can have Report Portal reports with Concept.js and Testomatio, the tool I'm working on is also tries to bring better reporting to your end-to-end tests. So reports are pretty important in nowadays testing because everything can be shown from a console and for sure your managers, your clients won't look to console to see what tests are passed, what tests not, for what reasons. They want a clean, nice graphical UI to understand this. This is why Concept.js provides so many option for reports because depending on your budget, depending on your conditions, you can choose whatever tool you want. By the way, all those three tools are absolutely open source. You can start working with them and with Concept.js today.

6. Important Features of Concept.js

Short description:

Concept.js supports parallel execution by workers, provides a solution for failed steps, and takes the traditional concept of page objects. It also offers Cucumber BDD support and the ability to communicate with your application via API. Concept.js focuses on end-to-end testing and is free and open source.

So let me sum up some important features of Concept.js. Not mention features of Concept.js because you already heard of switching between engines, interactive pause, easy to learn syntax. What else do we have? And yes, I was already showing this a bit. Maybe you have already seen it.

So we support parallel execution by workers. So very modern, very elegant API. You can write your own custom runners so you could parallelize your tests however you want. For instance, group run tests in two groups, in two browsers from Firefox or make four groups at Safari. So you can split tests however you want into groups, bring as many browsers you need and have parallel executions and have single report after this. That's really cool.

Also built in, and it's a simply working solution for failed steps. So automatically tries for steps. So if some element is not found on page on first try, you won't even notice it. Concept.js will retries this and wait for it to appear without even notifying you. So it works for any command, click, see, so whatever it is. So it just works, just don't touch it and you're fine.

Concept.js takes the traditional concept of page objects and it's built into the framework so you can easily refactor your code to make it easy to be used in other tests. So page objects, step object, data objects are all built in. And as I mentioned previously, we have Cucumber BDD support built on top of Concept.js. So this is really awesome because you can combine true BDD with specifications in JavaScript like Concept.js tests. This is important because not all tests should be written in the truly BDD manner because not all tests are features. So because you need to have regression tests for box. So my point is that you should combine BDD approach with classical approach and only Concept.js testing framework allows you to do this.

So another cool feature I really like is if you have API and you could communicate with your application via that API, for instance, you could create a fixtures through this API, delete fixtures. And it is much easier to do it through the REST API interface instead of web interface. So you can easily combine them together. And that's my final points. Concept.js is a testing framework and what's different from other, it focuses on end-to-end testing without executing your end-to-end test by itself. So it tries to take the best user experience into itself to make testing as easy as possible. So Concept.js is absolutely free, open source.

7. Contributing to Concept.js and CodeSapJS Features

Short description:

You can contribute to Concept.js. Different engines serve different purposes, allowing you to choose the right tool. Cypress is popular for its developer-friendly approach, while WebDriver is suitable for QA engineers who require more browser control. We have received questions about CodeSapJS's multi-browser mode compatibility, and it does support it. CodeSap delegates certain trade-offs to the underlying engines, making it more flexible than Cypress. If you need full browser control and advanced features, CodeSap js is a better choice.

You can contribute on it. There are already big and small companies using Concept.js and joining it. So join our team, join our community, try Concept.js on your own to check it and it works really great with Playwright. Thanks for watching, my name is Michael and see you soon.

Yeah, I thought that Cypress will take even greater numbers here. I'm pretty impressed that someone is using Concept.js. What I wanted to show with this polls that there are different engines and for different purposes, you can choose and the tools that fits right for you. So I think the Cypress is very popular because it's very developer friendly and it took on board lots of developers who didn't like testing, who didn't want to do this and browser testing seemed to them complicated because it was always a product of Selenium to deal with. And the part is WebDriver is more for QA engineers who work on end-to-end testing because they need more tool browser control. So I think it's like two audiences here and yeah, they're quite different but we need to engage all of them to collaborate and make better tests.

That makes a lot of sense. We've also started getting questions for your talk. So I've got a few questions for you, if that's okay. I'm gonna start with them and I'm not gonna mention the names of the people who have asked the questions because I'm gonna butcher the name so there's no point. But one of the first questions was, does CodeSapJS work in the multi-browser mode? For example, a pair of browsers are interacting with each other.

Sure, sure. We had this feature implemented like year ago, no two years ago. So yeah, it's quite some time. So it was working, it was better tested and yes you can use it. Or two different browsers if you want it.

Okay, okay. So that's really good. Somebody made the comment about CodeSap having different trade-offs like Cypress does. Cypress for example, says it doesn't have much tab support and you can't use it to drive two browsers at the same time. How's CodeSap fared compared to those?

CodeSap doesn't solve these problems. It delegates it to the engines that already solve this problem. So this is why I prefer tools like Playwright or Puppeteer which have full browser control, and this is why I dislike Cypress for most of my testing work. But yeah, this is user preference and whatever better fits to your job. So if you need to test single-page application with the single components, Cypress is really fine. But if you need full browser control and more and more features, CodeSap js is better because you can choose the engine while you write your tests.

8. CodeSap.js Strategies and Limitations

Short description:

CodeSap.js offers strategies to deal with slow-loading elements, including explicit waits, manual retries, and retrying the full scenario. There are no restrictions in CodeSap.js itself, as it delegates browser control to Playwright. However, if you need special APIs, you can write Playwright, Puppeteer, or WebDriver code. Recording interactions in a browser is not recommended, as the interactive pause mode in CodeSap.js provides more flexibility and effectiveness in testing.

So the code base is growing, but you're still able to choose a better engine if you need it. That makes a lot of sense and I like that, especially since people were really skewing towards one of the engines out there.

How does CodeSap deal with things that take a while to appear in the page? For example, a single-page app? Sorry, how does it, what? How does CodeSap deal with things that take a while to appear? So they're slow to load on the page because it's a big page, like a single-page application where everything kinda has to load before you can interact with it.

Yeah, so the problem with single-page application, we don't know is it loaded or not. So it never says us anything about it. So there is no state of this application. So we have like different strategies on this. The default strategy is outer, trying each step. So each step in the test will be automatically retried on failure. We have explicit waits. So you can wait for anything you want there. You can add manual retries. And if you have very, very bad applications that not very flaky, you can retry the full scenario. This is how it also works. I think that's all strategies, but maybe there are a few more of them. But this is that I use regularly. And that works.

People want to know more about the restrictions of using CodeSapp.js. Are there any restrictions like file uploading or working with iframe? What are some of the limitations of the framework? That's quite incorrect questions because I really feel it like Cypress auditory and you can't do it because CodeSapp.js is not driving the browser. It's delegated to Playwright and Playwright has no restrictions. So the only restriction you need to understand that CodeSapp.js can't do everything that Playwright can. So you need to write some Playwright code or Puppeteer code or web driver code if you need some very special APIs to be used. But yeah, you can do everything that you can do with Playwright, Puppeteer, web driver, et cetera, with CodeSapp.js.

Okay, that makes a lot of sense. That makes a lot of sense. Someone wants to know if you can generate CodeSapp.js test by recording interactions in a browser? I'm very skeptical about this idea. So I tried to use it, and I switched to this interactive pause mode. And I found this much more effective because I'm not just recording my actions, I'm testing it in real time, which I can't do with a recorder. So this is much more flexible and effective as for me as a tester. That's really good.

9. CodeSapp Auto-Reply Feature and Q&A

Short description:

When CodeSapp does the auto-reply feature, is that recording only on success on the first try, or for example, on the third try? The person asking the question has a concern about retries that can mask intermittent bugs. I would like to talk about it after my talking in the room. Please join Michael for more questions. Thank you, Michael, for a talk and for joining me for the Q&A. Thank you for inviting me. Thank you for the opportunity to be here with you.

There's a lot of questions coming in from Michael. We won't get to all of them. We'll take one more. So when CodeSapp does the auto-reply feature, is that recording only on success on the first try, or for example, on the third try? The person asking the question has a concern about retries that can mask intermittent bugs. So how does the auto-retry record happen? I think it's quite a technical question. I'm not sure I will fit in time to answer it, but I would like to talk about it after my talking in the room.

Oh, perfect. In that case, that's a perfect like for me to remind everybody that they can join you in your speakers room on The link to join is on the live stream timeline, and please join Michael for more questions. We won't get time to go into any more. Thank you, Michael, for a talk and for joining me for the Q&A. It was really good. Thank you for inviting me. Thank you for opportunity to be here with you.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

TestJS Summit 2022TestJS Summit 2022
27 min
Full-Circle Testing With Cypress
Cypress has taken the world by storm by brining an easy to use tool for end to end testing. It’s capabilities have proven to be be useful for creating stable tests for frontend applications. But end to end testing is just a small part of testing efforts. What about your API? What about your components? Well, in my talk I would like to show you how we can start with end-to-end tests, go deeper with component testing and then move up to testing our API, circ
TestJS Summit 2022TestJS Summit 2022
20 min
Testing Web Applications with Playwright
Testing is hard, testing takes time to learn and to write, and time is money. As developers we want to test. We know we should but we don't have time. So how can we get more developers to do testing? We can create better tools.
Let me introduce you to Playwright - Reliable end-to-end cross browser testing for modern web apps, by Microsoft and fully open source. Playwright's codegen generates tests for you in JavaScript, TypeScript, Dot Net, Java or Python. Now you really have no excuses. It's time to play your tests wright.
TestJS Summit 2022TestJS Summit 2022
21 min
Tiny Tests, Large Results
Yes, Big things do come in small packages. For example, isn’t a unit test’s speed, feedback, and reliability fantastic? Did you know we can also have fast, focused, and reliable feedback from our functional e2e tests? Atomic e2e tests are those that are targeted and focused. They’re tiny in size but large in their impact. This tutorial will teach you how to create atomic e2e tests with several code examples. First, we will use to authenticate by setting a cookie. Instead of using a UI. Second, we will use to set a JSON Web Token for authentication. Join me, and let’s write tiny tests for large results.
TestJS Summit 2022TestJS Summit 2022
18 min
Testing Mail Service With Playwright
We send emails to our users - account verification and newsletters. We allow the user to contact us by sending an email via inbuild form. Do we? Does the user receive an account verification email or exactly what notification they signed up for? We can cover this functionality as part of E2E tests: get an email and open it to check what is in it. We will need Playwright and a fake SMTP server to capture emails sent by the app.
TestJS Summit 2022TestJS Summit 2022
21 min
E2E Tests for Web3 Applications
We will go through a brief explanation of what is Web3 and the architecture of a web3 application. Then we will talk about how to end-to-end test, its challenges, some test tools that are available, and a demo using cypress and metamask.
Agenda: What is Web3; The Architecture of a Web3 Application; Web3 E2E Tests Introduction; Web3 E2E Tests Challenges; E2E Test Tools; Demo.
React Advanced Conference 2022React Advanced Conference 2022
30 min
How I Test a Million UI States with Every Merge — Visual Testing with Storybook
Error state, loading state, awkward breakpoint, bad data, poor formatting, browser support. Every component can result represent hundreds or thousands of discrete visual states. How do you test it? Manually disable the network — temporarily. Insert bad code — just for a minute. Paw at the edge of your screen. Hack local database fixtures to bits. Frontend development has so many dimensions. Time and variation result in an infinite number of UI possibilities. In this talk, we'll use Storybook to progressively develop, test, and document our work — automating the grunt work of UI development.

Workshops on related topic

React Summit 2022React Summit 2022
118 min
Detox 101: How to write stable end-to-end tests for your React Native application
Workshop Free
Compared to unit testing, end-to-end testing aims to interact with your application just like a real user. And as we all know it can be pretty challenging. Especially when we talk about Mobile applications.
Tests rely on many conditions and are considered to be slow and flaky. On the other hand - end-to-end tests can give the greatest confidence that your app is working. And if done right - can become an amazing tool for boosting developer velocity.
Detox is a gray-box end-to-end testing framework for mobile apps. Developed by Wix to solve the problem of slowness and flakiness and
used by React Native itself
as its E2E testing tool.
Join me on this workshop to learn how to make your mobile end-to-end tests with Detox rock.
- iOS/Android: MacOS Catalina or newer
- Android only: Linux
Install before the workshop
TestJS Summit - January, 2021TestJS Summit - January, 2021
173 min
Testing Web Applications Using Cypress
Workshop Free
This workshop will teach you the basics of writing useful end-to-end tests using Cypress Test Runner.
We will cover writing tests, covering every application feature, structuring tests, intercepting network requests, and setting up the backend data.
Anyone who knows JavaScript programming language and has NPM installed would be able to follow along.

TestJS Summit 2021TestJS Summit 2021
146 min
Live e2e test debugging for a distributed serverless application
Workshop Free
In this workshop, we will be building a testing environment for a pre-built application, then we will write and automate end-to-end tests for our serverless application. And in the final step, we will demonstrate how easy it is to understand the root cause of an erroneous test using distributed testing and how to debug it in our CI/CD pipeline with Thundra Foresight.
Table of contents:
- How to set up and test your cloud infrastructure
- How to write and automate end-to-end tests for your serverless workloads
- How to debug, trace, and troubleshot test failures with Thundra Foresight in your CI/CD pipelines

TestJS Summit - January, 2021TestJS Summit - January, 2021
128 min
Uniform Browser Automation Infrastructure
In this workshop, I will show you how to quickly deploy and use browser automation infrastructure with Moon solution. We will start deploying everything on your workstation and will soon be able to run Selenium, Playwright and Puppeteer tests in parallel in the same cluster. Then I will demonstrate how to easily deliver the same experience for your team using a remote cluster in the cloud platform.