Vitest Mobile (React Native)

[pzuraq]

Hello! I'm a big fan of Vite/Vitest, have been using them for years. Our company needed better React Native testing, so I made Vitest Mobile: https://github.com/pzuraq/vitest-mobile/

The thing that sets this apart from other projects like vitest-native and vitest-react-native is that Vitest Mobile actually boots a full emulator/simulator and runs the tests in the real environment. The idea is to use the same structure that Vitest Browser established and to try to fit into the ecosystem in exactly the same way. I also considered react-native-harness, but I really ended up believing that the Vite model works much better. The parts are just much more composable, and building it was relatively straightforward - all signs of a well run, well factored project.

Anyways! I wanted to raise a couple of topics for discussion here:

Upstreaming?

Would this be something we could eventually upstream into the Vite/Vitest org/umbrella? Would the team be interested in an "official" solution for testing apps in a mobile environment? I think, FWIW, we could also test PWAs with a lot of the same tooling, so this wouldn't necessarily be limited to just React Native apps, it really could be a set of composable tools for running Vitest in emulators/simulators in general.

Coordinating API development and providing feedback?

Even if it's never an official plugin in the Vitest org, I'd like to work with the team to make sure that everything is solid and does things the "Vitest way" internally. In my experience, that makes these projects much easier as a maintainer, using solid public APIs rather than hacking around things. The pool API already feels like it has some limitations that I'm working around, so would love to spend some time (eventually, when I have time 🫠) hardening those APIs and maybe working with y'all to figure out where the rough edges are.

Replacing Metro Entirely?

The biggest one here, and probably a wider discussion than just Vitest, but this is a good place to start.

So, Metro is very much based on a non-Vite like build world. It feels very tailored to React Native, and very built for the Webpack world. All of these assumptions are built in, and that basically means I have:

1. Scaffold a whole React Native app in a ~/.cache dir
2. Boot Metro in the background, and
3. Coordinate the Metro server with the Vite/Vitest server

This is the most brittle part of the architecture IMO, for a number of reasons. I also felt like, as I started to dig in, maybe we could just replace Metro with Vite?

For reference, I'm a former Ember.js core team member and worked a lot with Ember-CLI stuff back in the day, so I'm very familiar with module systems and AMD like output as a compile target. This is what Metro ultimately produces, since React Native apps generally just have one big JS file, and Hermes does not support import directly.

I feel like what we could do here is something like this:

1. Setup Vite in build mode to compile all of the files like normal, and run them through Hermes to generate byte-code. This might happen before or after bundling, but I feel like it should "Just Work" unless I'm missing something.
2. In dev mode, Metro just sends plain-text JS, it doesn't compile to byte-code, SO we could
   a. Setup an module system in the built output, similar to Metro
   b. Send updates over a WebSocket, eval them, and replace the modules, again, like Metro
   c. But, change any custom semantics for HMR and such to match Vite's model

This would all be done as a plugin, to be clear, not built directly into Vite. I think the plugin model would already be flexible enough here, for the most part. If we did this, we would be able to run Vite as the webserver, and Vitest Mobile would be able to use the same update patterns as standard Vitest (rather than having to rely on Metro HMR semantics).

Things that would be out of scope IMO would be:

1. TurboModules / native module compilation - that's something that we would rely on the existing tooling for, it's not really a concern of the JS build pipeline
2. The whole scaffold and setup for iOS/Android app projects. Again, that's just something that's not really in scope for Vite IMO

I won't have time to work on this in the immediate future, but I wanted to start the discussion to share ideas, and because maybe someone else does have that time. Also, maybe I'm underestimating the complexity of the Metro build pipeline, definitely still new to it so not sure if there are aspects I'm not thinking about.

[AriPerkkio]

This is really interesting! Great to see this implemented as custom pool. It kind of validates the idea we've been thinking about whether custom pools could work as abstraction for running test cases in non-Node environments. 💯

Process hanging after tests complete

The WebSocket server may keep the event loop alive. This is a known upstream issue with the Vitest custom pool API — there's no close() lifecycle hook to distinguish "file done" from "run done."

This is not a known issue. There is pool.close() that you should probably use for this.

[pzuraq]

Whoops, that's on me not double checking AI enough 😅 I had written up an issue description and wanted to double check it before submitting, and it decided to include it in the README it seems.

On that note though, is close an option supported on pool workers? Because currently, it's not in the types or the guides, and it doesn't have the same semantics as stop. I was going to dig in and play around more at some point to figure out if I could actually have parallel workers, or parallel connections to the emulators, etc. But yeah, would love to know the best way to do this.

[AriPerkkio]

close is a main process/thread side event for the whole pool. If it's a worker side event that's needed, then there is a teardown hook that's called when a single worker is about to be destroyed:

vitest/docs/guide/advanced/pool.md

Lines 135 to 137 in edacb0f

	teardown: () => {
	// Optional, provide a way to teardown worker, e.g. unsubscribe all the `on` listeners
	},

there's no close() lifecycle hook to distinguish "file done" from "run done."

From a single worker's perspective: stop and teardown would be the "file done" when isolation is enabled. If isolation is disabled, the end of runTests would indicate that your worker is finished with a single file.

vitest/docs/guide/advanced/pool.md

Line 144 in edacb0f

runTests: (state, traces) => runBaseTests('run', state, traces),

[pzuraq]

So, the tricky bit here is that start and stop are run for every test file/task when isolate: true, and with the current setup I have I don't think that really works.

Emulators/simulators are expensive, and inside of a React Native app there isn't really a way to cleanly create an isolated context. So, we generally want to:

1. Start up the emulator in the beginning (which can take a few minutes)
2. Open a connection,
3. Send the test files
4. Run them serially
5. Close the connection and cleanup the websocket

This is particularly weird when using Metro as the build tool, because Vitest actually doesn't control HMR, and that's where a decent amount of the complexity comes from.

I currently can work around this by setting isolate: false and maxWorkers: 1, but the pool doesn't control those. That's why I have to add a full Vite plugin and configure the config.

Will think about if there's a better way to structure this, but Vitest Browser seems to have some extra ability/lifecycle hooks that could really help her.