Question about fairness and starvation

[mzimbres]

Hi, I am a C++ developer that has contact with Rust and Tokio from time to time. I also happen to have interest in networking and have some experience with Boost.Asio. These days I was reading this article https://tokio.rs/blog/2020-04-preemption and this part raised my attention

If data is received faster than it can be processed, it is possible
that more data will have already been received by the time the
processing of a data chunk completes. In this case, .await will never
yield control back to the scheduler, other tasks will not be
scheduled, resulting in starvation and large latency variance.

Currently, the answer to this problem is that the user of Tokio is
responsible for adding yield points in both the application and
libraries. In practice, very few actually do this and end up being
vulnerable to this sort of problem.

Boost.Asio is not succeptible to this kind of problem because all asynchronous operation use default-rescheduling, this means regardless whether to socket is ready for reading the next operation must be rescheduled for execution. In this scenario there is no way for a task to monopolize the event-loop causing unfairness and starvation of other tasks as mentioned above. Has this ever been considered in Tokio?

Later on the article states

As of 0.2.14, each Tokio task has an operation budget. This budget
is reset when the scheduler switches to the task. Each Tokio
resource (socket, timer, channel, ...) is aware of this budget. As
long as the task has budget remaining, the resource operates as it
did previously. Each asynchronous operation (actions that users must
.await on) decrements the task's budget. Once the task is out of
budget, all Tokio resources will perpetually return "not ready"
until the task yields back to the scheduler. At that point, the
budget is reset, and future .awaits on Tokio resources will again
function normally.

This fixes the unfairness and starvation problem, but won't this create new problems? For example, I am thinking of a websocket server where all sessions share the same connection to the database, which can be seem as the bottleneck. If all DB operations on this connection are now treated as not ready all remaining operations are also slowed down, since now the DB connection is not as fast as it could.

It would be great if you could these questions or point me to docs explaining the design rationale.

Thanks

[Darksonn]

this means regardless whether to socket is ready for reading the next operation must be rescheduled for execution.

I'm not familiar with Boost.Asio, but from your description it seems like a much more aggressive version of what we're doing. We only force it to yield after 256 successful operations, whereas this sounds like it would happen after every individual operation?

If all DB operations on this connection are now treated as not ready all remaining operations are also slowed down, since now the DB connection is not as fast as it could.

If there are no other tasks on the runtime for execution, then the DB task will be scheduled again immediately. The time it takes to do that is insignificant compared to the cost of IO.

And if there are other tasks, then they get to run first. But in that case, this is what we wanted, because the DB task would otherwise be starving those other tasks.

Note that cooperative scheduling can be turned off for specific futures using unconstrained.

[mzimbres]

whereas this sounds like it would happen after every individual operation?

Yes. But the rescheduling time can be as low as 50ns so not an overhead compared to IO.

Thanks for your response.