How Realm by MongoDB is Testing Native Modules “on device”

Rate this content

Running tests on Node.js is common practice for app and library developers. But when your library is integrating more deeply with the platform, operating system or JS engine, you need to run tests “on device” increases. The Realm JS team at MongoDB is faced with this challenge, as we’re maintaining a React Native library with native modules. In this talk I’ll share the “mocha-remote” package which we’ve built to help us run tests “on device” while getting reporting and remaining in control from the comfort of our terminals.

11 min
15 Nov, 2023

AI Generated Video Summary

The speaker works on the Atlas Device SDKs, which allows testing code across multiple platforms and has support for multiple programming languages. They also built Mocha Remote CLI, a tool for running tests on Node.js and in a browser. The speaker mentions the popularity of Jest and the alternative Vitest for running tests on platforms like Android and iOS.

1. Introduction to Atlas Device SDKs

Short description:

I work for MongoDB on the Atlas Device SDKs, supporting multiple operating systems, platforms, and JavaScript engines. Today, I want to share a technique we use to test our code anywhere it needs to run. MongoDB Atlas is a multicloud developer data platform, enabling you to build with data. We built the Atlas Device SDKs, which features a local database running in the app and bidirectional synchronization of data between the app and the cloud. We have support for multiple programming languages and leverage code generation to expose a common core component. We want to test across all supported platforms and have an awesome developer experience.

Hi there. I'm Krane. I work for MongoDB on what we used to call Rel, but is now known as the Atlas Device SDKs. I help build and maintain our TypeScript package, supporting multiple operating systems, platforms, and JavaScript engines.

And today, I want to share a technique we use to help test our code anywhere it needs to run. MongoDB Atlas is a multicloud developer data platform, an integrated suite of cloud database and data services, enabling you to build with data. And one part of this is about bringing data closer to where it's being used. And we call this Atlas for the Edge.

One place where your data is being used is in your end users mobile devices. And for that, we built the Atlas Device SDKs, formally known as Realm. So the Atlas device SDKs features a local database running in the app. It has bidirectional synchronization of data between the app and the database running in the cloud. And it doesn't require internet connectivity, unless it needs to synchronize, of course. And because it doesn't have this requirement, when you read and write data, it's very low latency. And besides those APIs, we also have APIs to observe data as they change on the server and propagates to the device. And this enables you to build truly reactive user experiences. Besides TypeScript, we have support for Swift, Kotlin, .NET, just to name a few.

And for that to happen, we have a common core component written in C++. And our team leverage code generation to expose this core component as a TypeScript interface on which we build the SDK on top of. And for that to happen, we generate code, glue code, binding code, we call it, between this core component and the various JavaScript engines and their C++ APIs. We currently support JavaScript core at Hermes via JSI for React Native and we also support V8 via NAPI for Node.js and Electron. We also have a preview of Wasm support for the browsers. So we have all this handcrafted TypeScript code and rules that we use to generate code for all these different platforms with JavaScript engines that behave slightly differently.

So we want to naturally test this across all the supported platforms before releasing something into the apps of thousands of users. When I joined, we had a single package exporting multiple async functions and we had specific test environments that were handcrafted code, some of it native, that wasn't tested at all. It was a nightmare to maintain, especially as we wanted to progress and update these environments to the newest versions of the of the platforms we supported. And this lack of developer experience grew on me. I just wanted to be able to run the test in watch mode on any platform that we supported, have an awesome developer experience, instead of spending my time upgrading these platform runners or platform-specific running code. And I thought to myself, what if we just ran the same code across all the platforms, and we drove it from the comfort of our terminals, and we could write and maintain, and more importantly, test this code separately from the environments in which we tested. I essentially just wanted to run a CLI on my host machine, execute the tests on device, and get the reporting back on my host machine.

2. Building Mocha Remote CLI

Short description:

In 2019, I built Mocha Remote, a tool based on Mocha for running tests on Node.js and in a browser. It supports almost all options of Mocha, including timeouts, specifying reporters, and a context. The Mocha runner was bundled into a runtime independent variant using webpack, and a WebSocket connection is used between the runner, client, and Mocha Remote CLI server. The CLI can grep for specific tests and supply a context. The Mocha Remote CLI is linked to a sub-command responsible for starting the test app, ensuring correct status code propagation.

So in 2019, that's exactly what I built. I built this around Mocha. It honestly probably didn't do too much investigation. It looked popular, and I knew that it could run tests both on Node.js and in a browser. And it was used elsewhere in our organization.

And the Mocha CLI, basically, this is a slide about the simplified architecture of Mocha. Basically, once you execute the CLI, it instantiates a runner, which requires in the test files, and then it starts execution. And the progress of passing and failing tests are emitted through events back to the CLI, which is then relayed to the reporters. And I named my tool Mocha Remote based on this. It's not as popular as Mocha as you can see, but and mostly it's 900 weekly downloads. Most of them are probably our CI. And this is part of why I want to share this story today.

I tried reusing the Mocha CLI, but I found it difficult to extend because the runner itself is not extendable or it's not possible to configure from the CLI of Mocha, but fortunately it was very easy to implement, re-implement that, and the Mocha Remote CLI now supports almost all of the options that you can pass to Mocha, including grabbing for a test title, including timeouts, specifying reporters, watching, and then also a context, which I will show later.

So the Mocha runner, the original Mocha runner, I was actually able to bundle that into a runtime independent variant using webpack, and this is what runs on the various platforms. Instead of just simple events, using event emitting, we use a WebSocket connection between the runner, client, and the Mocha Remote CLI server. And once the client connects to the server, it will get a message to start running tests. The client or the runner will execute a test function supplied by the developer, which will basically define the tests either by importing them or just defining them inline. And once the test runner starts running, it will start sending these passing and failing tests events over a web socket back to the CLI, which is propagated to the reporters.

This is what it looks like when you invoke the Mocha Remote CLI, and you can see we grep for all the tests that has connects in the title, and we also supply this context, in case an API URL, which is propagated through this system into the runner, and I'll show this in a second. We also supply the sub-command noderuntestappjs, which is responsible for starting up the test app. And the lifetime of this sub-command and the Mocha Remote CLI is linked together in a way that if the sub-command exits prematurely, the Mocha Remote CLI will also exit. And by and vice versa. So it also ensures that the correct status code is propagated when the tests are failing in the device or the device crashes or whatever, the Mocha Remote CLI will get the correct non-zero exit code.

Cool. I believe this is almost time for demos. I just want to show here, this is what it looks like in a client. It gets the test, or you supply the test function and it gets a context path, this argument, and you can see how this context is able to be used when you instantiate this MyApp instance. Cool. I want to show first running, invoking the Mocha Remote CLI here on node.js. And this starts up a separate node process that executes the tests.

3. Running Tests on Android, iOS, and Other Platforms

Short description:

We can also do it for Android, iOS, and other platforms. Maka was popular in 2016, but Jest is now the de facto standard in React Native projects. I've started the Jest remote package to support running tests in a separate Node process. Another alternative, Vitest, is gaining attention from the React web community. It has an experimental browser mode and a platform-independent runner.

We can also do it for Android here. It will basically invoke the Metro, the metro bundle as one part of it The other part of the sub-command will execute the React Native CLI, and you can see the emulator will start up and starts requiring in the bundle and execute the tests. And thankfully it passed, which is great. I don't want to spend time, but we can also do it for iOS.

Yeah. Let's go back to the slides. Good. So, Maka was popular in 2016 when we started the Realm.js Project. Arguably it's still popular with 7 million weekly downloads, but not as popular as Jest. And Jest is the de facto standard in React Native projects. And this is why I've also started the Jest remote package. It's built around the same architecture and it's a good match because the Jest runner is actually pluggable from the CLI. So it's possible to specify this Jest remote runner. And currently, in its current form, as I'm recording this, it supports running the tests in a separate Node process. But my vision is to, of course, support all the other platforms that we also support with Maka Remote.

Another competitor, or you can say an alternative to Jest, which is catching a lot of attention from the React web community especially, is Vitest. And from my initial investigations and conversations with maintainers, it looks like this will be an even better match as it also has experimental browser mode. So the runner is already built in a way that's platform and runtime independent.

Thank you so much for your time and attention. This is a rare commodity these days. And I hope that you found this useful, especially if you too as a maintainer of a package like this. And if you have any questions, feel free to reach out. All my social media handles are in my Git Nation profile. Also, please help me build this for Vitest or Jest if you want to, if you have time and thinks it's interesting. I wish you the best of luck with all of your endeavors.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

TestJS Summit 2021TestJS Summit 2021
33 min
Network Requests with Cypress
Whether you're testing your UI or API, Cypress gives you all the tools needed to work with and manage network requests. This intermediate-level task demonstrates how to use the cy.request and cy.intercept commands to execute, spy on, and stub network requests while testing your application in the browser. Learn how the commands work as well as use cases for each, including best practices for testing and mocking your network requests.
TestJS Summit 2021TestJS Summit 2021
38 min
Testing Pyramid Makes Little Sense, What We Can Use Instead
Featured Video
The testing pyramid - the canonical shape of tests that defined what types of tests we need to write to make sure the app works - is ... obsolete. In this presentation, Roman Sandler and Gleb Bahmutov argue what the testing shape works better for today's web applications.
TestJS Summit 2022TestJS Summit 2022
27 min
Full-Circle Testing With Cypress
Cypress has taken the world by storm by brining an easy to use tool for end to end testing. It’s capabilities have proven to be be useful for creating stable tests for frontend applications. But end to end testing is just a small part of testing efforts. What about your API? What about your components? Well, in my talk I would like to show you how we can start with end-to-end tests, go deeper with component testing and then move up to testing our API, circ
TestJS Summit 2021TestJS Summit 2021
31 min
Test Effective Development
Developers want to sleep tight knowing they didn't break production. Companies want to be efficient in order to meet their customer needs faster and to gain competitive advantage sooner. We ALL want to be cost effective... or shall I say... TEST EFFECTIVE!But how do we do that?Are the "unit" and "integration" terminology serves us right?Or is it time for a change? When should we use either strategy to maximize our "test effectiveness"?In this talk I'll show you a brand new way to think about cost effective testing with new strategies and new testing terms!It’s time to go DEEPER!
TestJS Summit 2023TestJS Summit 2023
21 min
Everyone Can Easily Write Tests
Let’s take a look at how Playwright can help you get your end to end tests written with tools like Codegen that generate tests on user interaction. Let’s explore UI mode for a better developer experience and then go over some tips to make sure you don’t have flakey tests. Then let’s talk about how to get your tests up and running on CI, debugging on CI and scaling using shards.

Workshops on related topic

React Summit 2023React Summit 2023
151 min
Designing Effective Tests With React Testing Library
Featured Workshop
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents- The different kinds of React application tests, and where component tests fit in- A mental model for thinking about the inputs and outputs of the components you test- Options for selecting DOM elements to verify and interact with them- The value of mocks and why they shouldn’t be avoided- The challenges with asynchrony in RTL tests and how to handle them
Prerequisites- Familiarity with building applications with React- Basic experience writing automated tests with Jest or another unit testing framework- You do not need any experience with React Testing Library- Machine setup: Node LTS, Yarn
TestJS Summit 2022TestJS Summit 2022
146 min
How to Start With Cypress
Featured WorkshopFree
The web has evolved. Finally, testing has also. Cypress is a modern testing tool that answers the testing needs of modern web applications. It has been gaining a lot of traction in the last couple of years, gaining worldwide popularity. If you have been waiting to learn Cypress, wait no more! Filip Hric will guide you through the first steps on how to start using Cypress and set up a project on your own. The good news is, learning Cypress is incredibly easy. You'll write your first test in no time, and then you'll discover how to write a full end-to-end test for a modern web application. You'll learn the core concepts like retry-ability. Discover how to work and interact with your application and learn how to combine API and UI tests. Throughout this whole workshop, we will write code and do practical exercises. You will leave with a hands-on experience that you can translate to your own project.
Remix Conf Europe 2022Remix Conf Europe 2022
195 min
How to Solve Real-World Problems with Remix
Featured Workshop
- Errors? How to render and log your server and client errorsa - When to return errors vs throwb - Setup logging service like Sentry, LogRocket, and Bugsnag- Forms? How to validate and handle multi-page formsa - Use zod to validate form data in your actionb - Step through multi-page forms without losing data- Stuck? How to patch bugs or missing features in Remix so you can move ona - Use patch-package to quickly fix your Remix installb - Show tool for managing multiple patches and cherry-pick open PRs- Users? How to handle multi-tenant apps with Prismaa - Determine tenant by host or by userb - Multiple database or single database/multiple schemasc - Ensures tenant data always separate from others
React Summit 2022React Summit 2022
117 min
Detox 101: How to write stable end-to-end tests for your React Native application
Compared to unit testing, end-to-end testing aims to interact with your application just like a real user. And as we all know it can be pretty challenging. Especially when we talk about Mobile applications.
Tests rely on many conditions and are considered to be slow and flaky. On the other hand - end-to-end tests can give the greatest confidence that your app is working. And if done right - can become an amazing tool for boosting developer velocity.
Detox is a gray-box end-to-end testing framework for mobile apps. Developed by Wix to solve the problem of slowness and flakiness and used by React Native itself as its E2E testing tool.
Join me on this workshop to learn how to make your mobile end-to-end tests with Detox rock.
Prerequisites- iOS/Android: MacOS Catalina or newer- Android only: Linux- Install before the workshop
TestJS Summit 2023TestJS Summit 2023
48 min
API Testing with Postman Workshop
In the ever-evolving landscape of software development, ensuring the reliability and functionality of APIs has become paramount. "API Testing with Postman" is a comprehensive workshop designed to equip participants with the knowledge and skills needed to excel in API testing using Postman, a powerful tool widely adopted by professionals in the field. This workshop delves into the fundamentals of API testing, progresses to advanced testing techniques, and explores automation, performance testing, and multi-protocol support, providing attendees with a holistic understanding of API testing with Postman.
1. Welcome to Postman- Explaining the Postman User Interface (UI)2. Workspace and Collections Collaboration- Understanding Workspaces and their role in collaboration- Exploring the concept of Collections for organizing and executing API requests3. Introduction to API Testing- Covering the basics of API testing and its significance4. Variable Management- Managing environment, global, and collection variables- Utilizing scripting snippets for dynamic data5. Building Testing Workflows- Creating effective testing workflows for comprehensive testing- Utilizing the Collection Runner for test execution- Introduction to Postbot for automated testing6. Advanced Testing- Contract Testing for ensuring API contracts- Using Mock Servers for effective testing- Maximizing productivity with Collection/Workspace templates- Integration Testing and Regression Testing strategies7. Automation with Postman- Leveraging the Postman CLI for automation- Scheduled Runs for regular testing- Integrating Postman into CI/CD pipelines8. Performance Testing- Demonstrating performance testing capabilities (showing the desktop client)- Synchronizing tests with VS Code for streamlined development9. Exploring Advanced Features - Working with Multiple Protocols: GraphQL, gRPC, and more
Join us for this workshop to unlock the full potential of Postman for API testing, streamline your testing processes, and enhance the quality and reliability of your software. Whether you're a beginner or an experienced tester, this workshop will equip you with the skills needed to excel in API testing with Postman.
GraphQL Galaxy 2020GraphQL Galaxy 2020
106 min
Relational Database Modeling for GraphQL
In this workshop we'll dig deeper into data modeling. We'll start with a discussion about various database types and how they map to GraphQL. Once that groundwork is laid out, the focus will shift to specific types of databases and how to build data models that work best for GraphQL within various scenarios.
Table of contentsPart 1 - Hour 1      a. Relational Database Data Modeling      b. Comparing Relational and NoSQL Databases      c. GraphQL with the Database in mindPart 2 - Hour 2      a. Designing Relational Data Models      b. Relationship, Building MultijoinsTables      c. GraphQL & Relational Data Modeling Query Complexities
Prerequisites      a. Data modeling tool. The trainer will be using dbdiagram      b. Postgres, albeit no need to install this locally, as I'll be using a Postgres Dicker image, from Docker Hub for all examples      c. Hasura