Why does my program get stuck? Unexpected interaction between two tokio runtimes

[LoneMartian]

Hi all,

Async Rust beginner here. The sample code in my repository contains generated gRPC code for a very simple Greeter service. The macros in src/lib.rs are supposed to provide a blocking wrapper around a single-threaded tokio runtime. The test in tests/integration_tests.rs creates a new thread and runs the Greeter service on a second tokio runtime. Subsequently, the main thread instantiates a client and calls two functions on it.

Running cargo test never returns, as some of the futures appear to get stuck. Here a screenshot from tokio-console:

Some general observations:

• When refactoring client and server into separate binaries (e.g. src/client.rs and src/server.rs), the programs behave as expected.
• When using two multi_thread tokio runtimes, the integration test behaves as expected.
• When moving the server from the integration test (e.g. implementing it in Python), the test passes with the current_thread runtime.

I hope anyone can point out to me what's going on here.

[Darksonn]

Tasks on a current_thread runtime only run if you are actively calling Runtime::block_on on it. If you aren't doing that, then that's probably the cause.

[LoneMartian]

block_on is called here for the client, as well as here for the server.

[Darksonn]

Yes, but perhaps there are still tasks on the runtime when you return from block_on? Those tasks will stop running until you call block_on again.

[LoneMartian]

You seem to be correct indeed. If I call block_on again, e.g. in drop, the program behaves as expected. Now I need to understand this behavior. The doc of block_on states the following:

This runs the given future on the current thread, blocking until it is complete, and yielding its resolved result. Any tasks or timers which the future spawns internally will be executed on the runtime.

I guess the second sentence could explain my observation? Internally, new tasks are put onto the runtime, which are blocked from completing because the initial future has completed and I have returned from the block_on call.

[Darksonn]

Yes. A current_thread runtime does not spawn its own threads, so if you're not calling block_on, then it doesn't have anywhere to run the tasks.

[LoneMartian]

What still puzzles me is how the future that I'm waiting on (e.g. the RPC call) can complete (as stated by the block_on documentation) while the tasks it has spawned internally have not.

[Darksonn]

Yes, that does sound surprising. But it sounds like that's just how the library works.