tokio::spawn with yield_now sometimes fail to wake up

[bobyangyf]

Not sure if tokio should guarantee this, or if its out of the contract, but...

If I spawn X task on a multi-threaded runtime with X worker threads, I would expect that all X of these tasks to be picked up and ran in parallel by the next available worker. That would mean that if the X tasks blocked these worker threads, and guarantee there is no additional spawns of any sort, eventually I would end up with all X tasks being ran by worker threads and hence blocked.
So, a verification of this can be done by using a sync Barrier of X.
Now this all works fine, until I add a yield_now to the spawned task.

With the yield_now, occasionally, a spawned task will fail to wake up despite the worker threads being idle (verified via thread dump). The repro isn't consistent, so it seems like there's some race in the tokio worker scheduling.

See playground.
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=64b6294b763d600d897b72cefebf8b41

May take a few reruns to get a deadlock and hence a timeout.

[Darksonn]

Work stealing relies on the non-idle threads making progress in some cases.

[wathenjiang]

The consistency in repros ask you use async thread syschronization tools , sync tools are beyond the control of tokio task scheduler.