Custom tokio runtime

[kvinwang]

Dear tokio developers,

It's great to see the features rt,time,... are stablized on WASM. However, there is still a gap to use it in our scenario.

Background

We are building a WASM-based app engine that runs inside TEE environment (Currently in Intel SGX). Where the wasm host runtime runs in TEE and the guest app is able to access the network via some async IO API. Because the threads in TEE are expensive, we designed the wasm runtime running in a fully async way.
That means, we impl Future for VmInstance. So a WASM guest is running via VmInstance::poll calls rather than calling a oneshot long blocking fn main(). This allows it to run much more WASM instances than giving each instance a thread to block in the guest main.

The problem

The current tokio runtime/driver for WASM tokio::runtime::scheduler::current_thread::Spawner is based on park/unpark. This means it would park the current thread while the app is idle. However, since our entire WASM guest is poll based, it would panic to park the current thread in the guest.

Idea about the solution

Is it possible to make the tokio runtime poll based rather than parking the current thread?
Or simply open a door to allow the user side to implement custom runtime for tokio? Something like:

#[derive(Debug, Clone)]
pub(crate) enum Spawner {
    CurrentThread(current_thread::Spawner),
    #[cfg(all(feature = "rt-multi-thread", not(tokio_wasi)))]
    MultiThread(multi_thread::Spawner),
    Custom(Box<dyn Spawning>),
}

The latter is preferred because we can delegate the guest tokio::time::Sleep to the host tokio::time::Sleep in this way.

More info

We have already successfully run some hyper-based HTTP server with our tiny custom async runtime and custom TcpSteam in the WASM guest. However, there are lot's of crates use tokio::spawn and tokio::time, we have to modify them to use sidevm::spwan and sidevm::time to run in our WASM guest.

[Darksonn]

This is not something that has a simple solution.

You cannot just provide a Box<dyn CustomRuntime> trait object because what you actually need is to have Tokio return from its entry point by returning every time it becomes idle, whereas Tokio will currently call a sleep function of some sort when it becomes idle. Calling a function and returning from a function when the runtime becomes idle are quite different behaviors.

[kvinwang]

We don't have to delegate the reactor and IO driver. Because the native runtime could have its own reactor and async io types, such as TcpStream, that implements tokio's (and futures's) AsyncRead/AsyncWrite. Well-designed libs in the ecosystem, such as hyper, would accept T: AsyncRead or T: AsyncWrite in their code base. So most code for IO could be runtime agnostic. But for the spawn and timer, there aren't such well-defined traits for lib authors to write runtime agnostic codes. They usually choose to use tokio's spawn and time API. So, it would be helpful if tokio allow a custom runtime just to hook those 2 APIs.

[Noah-Kennedy]

I'm not entirely sure if this is the write way to go about this. I've been working on an abstraction layer for this sort of issue, and I think that having a factory trait that let's you spawn, access timing APIs, and construct IO types is probably the best way to go about this, and it wouldn't require changes in tokio to do.

[kvinwang]

I think that having a factory trait that let's you spawn, access timing APIs, and construct IO types is probably the best way to go about this, and it wouldn't require changes in tokio to do

Yes, it would be nice if there are such fundamental traits somewhere for lib authors to implement. Sounds like it would take a long time to stabilize the API and wait for the ecosystem to adapt to it.

Since the tokio's spawn/timing API has already been widely used in the current ecosystem, could we have a temporary solution for the issue?

[Noah-Kennedy]

This would basically force us to stabilize and expose a lot of internals though, which is something I am really not keen on doing, especially with this area of the code. The problem here is that while this might be a temporary fix for folks in the ecosystem, it is a permanent change for us with lasting consequences for our internals. I would say that the proper approach is to get a compatibility layer in place for runtime spawning. Even if that takes time, it's still far better than this.

[kvinwang]

Fair enough. Thanks.

I've been working on an abstraction layer for this sort of issue.

Is there some link I can follow up on your work?