Hi, you're watching TestJS Summit, and this is the Lightful integration tests with test
containers.
testing is very important. More projects should test applications better. And I hope after this quick session, you're going to learn about how you can do integration tests, which you like using test
containers libraries. My name is Alex Salaev, and I work as a developer relations person at Atomic Jar, a company created, a startup created by the test
containers Java maintainers originally. And now we have more people from different language ecosystems helping us work in test
containers. If you have any questions, you can find me online. I'd be happy to chat about anything, test
containers or
testing related, or just software engineering in general. I think it would be very, very cool. So drop me a line.
testing is super, super important because it lies on the critical paths from development to production. If we don't have a good automated test suite, we cannot release things well. We need to have automated tests because we want to make sure that whenever we have something that we potentially want to release, we can go through our pipeline without bottlenecking on any manual process. This is helpful during a normal development practice development loop, but it's also super helpful in case there are any
security issues or supply chain
security issues where you update the third party packages and then you need to release things because they could be
security vulnerability fixes. But if you don't have a good test suite that you trust, then this is a manual process and you are as good as exposed. But if you do, you can run your automated tests, you can release immediately because you have confidence in your tests. This is very, very important. And lately, the way how we see what types of tests we want to run has been shifted. In the past, we had the
testing pyramid and we ran a ton of unit tests and they covered all possible scenarios and we had very good test coverage and then it still missed some issues. So recently, independent teams have been coming out how they're rethinking the
testing pyramid and how they put more and more emphasis on integration tests. Thinking about that, it makes a lot of sense. Our applications have become smaller. We are mostly writing and we are talking about better
backend applications here. We are mostly writing
microservices that talk to other APIs or talk to various technologies like databases or message brokers or
cloud technologies. And the application behavior very much is encoded in the interactions with those third party systems rather than the business logic within the particular application, how it transforms the
data. So it does make sense to have fewer implementation detail tests and use the integration tests which run your application with the immediate environment, with all the necessary components for the application to run properly as it would run in production, but in your
testing setup. That could be the bulk of our test suite. That could be the test that we trust and rely on. And we still can have end-to-end integrated tests that run in the environment, similar to production, where all the systems are spin up at the same time. And when we check the actual workflows as if that would be a production environment or maybe production
data or similar
data like that, but in a much larger environment. So for a test suite that you run everywhere on your machine, on your colleagues' machine, in your CI, integration tests hit the sweet spot between the simplicity of the setup and also how many issues with the real-world technologies they can catch. That's why they are getting more and more popular. This brings us to test
containers. Test
containers is libraries in different languages, including the test
containers node implementation that works for
javascript and
typescript. They integrate with Docker to create ephemeral environments where you can run all the third-party service dependencies that your application requires for
testing. You can run the databases, you can run your Kafka, you can run your Elasticsearch, you can learn your local stack if you work with LWS technologies. You can run them in Docker
containers and your application has the full control over the lifecycle of those. And your tests have the full control over the configuration of those. So you can test your application with the real dependencies and know that it works as expected. Test
containers has recently been named in the ThoughtWorks technology radar. It was put into the adult category, which means technically that there should be a strong reason. You should really know what you're doing if you don't want to use test
containers. They are allowing, test
containers allows you to create a reliable environment with the programmatic creation of those lightweight
containers for your dependencies. And it makes your tests more reliable and it tries to nudge you into doing the right things with your integration tests. And that's why there are more and more projects which are using test
containers in various setups and environments. Test
containers uses Docker as the environment where it spins up those
containers that your application wants to run. And this is great because Docker is almost universally available. It runs on all popular operating systems and developers understand how Docker works or how to use Docker from the outside. So this is a great, great option for leveraging a runtime to run those dependencies for your application. However, the stock sort of look and feel, the
user experience of Docker is not sometimes flexible enough for your integration tests. Docker is great because it has all the software in the world that can be run in Docker. There are registries where you can pull all the technologies that your soul requires. It provides you with the process isolation. It provides you with the ability to configure both the container and the application within the container to give you the CPU and memory limits. All those good things. But it is a little bit inflexible for the tests specifically because during the tests we want to put our application into the specific scenarios where something might go wrong. What will happen when the application works with a
database and the
data schema is incorrect? Or what will happen if my application has a long latency until it reaches Kafka? What happens when my Redis key numbers are close to the integer range and are trying to overflow? All the different scenarios and they all break the setup in some way. This is the notion of tests. This is what tests should do. They put your application under stress and then they want to figure out whether it behaves correctly. So with Docker, once you break the environment, it's very, very hard to recreate the environment. And this is where Test
containers comes in. Test
containers gives you the programmatic access to create, manage, lifecycle, and clean up the
containers that you want to run. It gives you
api to configure both the container, like expose which ports you want to expose. From the container, if you're working with it through the network, or which files you want to copy, or whether you want to programmatically follow the logs of the container and so on. So you can configure everything from your application tests, from your IDE, and you can do this any number of times. So tests bring their own environment into the play. And it also integrates with various
frameworks and test libraries. For example, there is a module for Jest Test
containers, which simplifies working with Jest tests and test
containers where you can declaratively specify which
containers you want, for example. Test
containers Node, as the other Test
containers implementations, is an open source project. Christian is a true hero of the Test
containers Node implementation, the main maintainer currently. There is an
npm package, which is how you get Test
containers into your application. And what it does, it uses Docker ODE to talk to the Docker environment. So your Docker environment doesn't need to be any particular Docker implementation. It of course runs with Docker Desktop, but it also can run with any other compatible Docker implementation. So for example, if you're running Minikube, the lightweight
kubernetes cluster, which exposes Docker
api, you can use that to run your Test
containers-based tests. Or if you're using a remote Docker, your Test
containers can talk to that. And internally at Atomic Jar, we're building the
cloud solution where you can get on-demand VM and run your Test
containers tests against that. So it's a very, very flexible setup, and it works really, really well. One thing that is very important here is that Test
containers takes care of the cleanup of the
containers. We know that for reliable integration tests, you need to have a repeatable environment. And for that, you want always to clean up after the run. That means if your tests pass, we clean up the
containers and remove them. If your tests fail, we clean up the
containers and remove them. If your machine runs with a remote Docker environment and your machine crashes and like internet blows up, we still will clean up the
containers on the remote Docker host. That means that you will never be in a situation where your test connect to the Kafka instance that you started two weeks ago, and it's lingering for some reason on your beefy CI machine. And then because the issues that will arise from that are really, really hard to reproduce and incredibly hard to
debug and fix. So Test
containers libraries try to nudge you into the right direction with Test
containers tests to enable parallelization of tests nicely, to kind of nudge you into using the correct
api to do the cleanup at all times. And in general, it's a very, very popular approach. Besides being just good library by itself, Test
containers comes with
ecosystem of the
modules where popular technologies have little implementation, little libraries, little
modules which specify and encode how to run that particular technology in your code. So you don't have to figure out what you need to do to run Cassandra in the Docker container or Kafka in the Docker container, but you can just use the
api and specify, give me a Kafka container, give me a
mongodb container, and you will get an instance of that immediately for yourself, which is great because that allows you to concentrate on the actual business logic of your tasks without spending time on figuring out the infrastructure, because that is managed by Test
containers. And it's not just a Node project, right? Test
containers is a good integration test are required in any
ecosystem of any programming language. So what you can do, you can have the similar approach in your Java applications, in your .NET applications, in your Go application, there's Test
containers Python, there is Test
containers rust. So it's a very, very popular and generic approach. And now I would like to show you a little bit how it feels to have tests and what are the building blocks of the
api that you need to know to be productive with Test
containers. Let's look into the IDE. So I have a very, very simple project here. It doesn't even have the actual application. I just want to give you the taste of the
api and talk through what's important from the Test
containers point of view. So we can just declare the dependency as normal to get that
npm package. And here in our test file, what we can do, we can require Test
containers and the basic building block is generic container. Generic container is an attraction that represents the container that we can manage via Test
containers. What we need to give it, we need to give it the Docker image name, which is the, in the current cases, Redis. We're going to run with the Redis container and then we do the configuration. We can expose the ports, we can configure which files we need to copy into the container. For example, this is a good way to instantiate your
database schema by sending it into your container. And then of course, we have the methods to use and configure the life cycle of our
containers. Of our infrastructure that we need for the tests. So here in this, in this just test, we are specifying that we want to create the container before all the tests. So this container will be shared between all the tests, which currently we don't have many, but we create the container. And then what, this is the interesting bit, the container can be run anywhere. It can be running on the remote Docker host or on your local host or in a VM somewhere. So when we create, when we make sure that our application knows how to talk to the technology, to the
database, or in this case, Redis in that container, we don't hard core any configuration, but we use that generic container instance to provide information where it's running so we can configure our application properly. So here we just say, oh, Redis, you're going to talk to, you're going to talk to Redis, the Redis client, sorry. You're going to talk to Redis by that host, which we get from the container and the exposed port 6379 will be mapped to the high level of random port on the host side here. So let's say, get the map port for that value. And then our Redis client is ready to connect to Redis. After all, we clean up everything, but we don't have to, at least we don't have to clean up the container because test
containers will, when our tests are passed, test
containers will clean it up by itself. But this is still a good practice. If you want to spin out like thousands of
containers during your test suite, maybe you should take care of the lifecycle yourself and stop them so they just don't use all the resources at the same time. So the test itself is super simple. We just want to put the values into the Redis and get the values in Redis. And you can see that if I run my application, then we can see that the
containers are there. So if I did Docker stats, there's currently only the root container, which is the container test that is used internally to clean up for the cleanup purposes. So if I just do a very simple setup here, so I will run tests again, and you can see that the
containers will be spinning up for a second and then running as well. So I don't run my Docker locally here. I use Task Manager's
cloud, which is connected to a cluster nearby. That's where my Docker
containers are running. And then if you want a larger example of our situation, then I have a different one. This is an actual Express app that uses
mongodb as a storage. And it just the test, very simple. Let's look at the test. We just use super test to send a request, actual high level functional tests for our application. And then we expect those requests to succeed, and then we expect the
data back. So this is the very high level functional test. And the interesting part here is that we don't build the
containers ourselves. We don't say how to configure
mongodb. But what we're doing, we're just importing the module that we can have. And if you look at the repository there, you can see that there are some
modules that are for fairly common technologies, MySQL, Mongo, Kafka, Elasticsearch, Postgres, that are there. And it's not such a large implementation. So if you're interested in any other technologies that you want to run, maybe after figuring out for your environment, consider contributing a module back. That'd be a great use. So here we use Mongo, and you can see how it works. So
mongodb container, we run. We wait for that. The test container's node is idiomatic
node.js package. So async await is supported, of course, for all long-running complex operations, like starting
containers and waiting for the technology in the container to start. And then we configure our
mongodb client to use the connection string from our Mongo. You don't even need to know what exactly is the connection string or how it looks, what's the format of that. You just can run it. And this is very, very great. So now after that, we, of course, disconnect. And then we can run the test, and here we run more tests. So if I run
npm test here, you can see in the background, if the Docker image is not in the cache of my Docker daemon, then the image will be automatically pulled from Docker Hub or, like, DeskCondenter supports the private registries as well,
authentication, anything that you can pull with Docker would be supported. And then you can just run it. And then after everything is said and done, the
containers are cleaned up and removed together with volumes and all of that. So it's a very, very flexible approach. It's very easy to integrate in your applications. You can also package your application in a Docker container, but I would prefer running it normally on my machine from my IDE, because then I can send the breakpoints and run it at will. So there's just one more slide. There is the, you can do the complex things. You can build complex topologies with network. You can wait for the
database in the
containers to start. You can create images on the fly. You can pull the logs and copy files back and forth and run the commands. Anything that works with Docker works with test
containers. And I think this is a really, really great approach. And you are welcome to check out the source on GitHub. And if you have any questions or if you want to try and talk to the
community, please join the Slack at slacktestcontainers.org to talk to the like-minded individuals. This is it. Thank you very much for watching. And if you have any questions, I'd be happy to answer them. Thank you.
