Tokio runtime: CPUs get maxed when serving many requests #2915

rosscanning · 2020-10-06T06:02:17Z

rosscanning
Oct 6, 2020

Version
Output of cargo tree | grep tokio:
tokio_cpu_demo v0.1.0 (D:\src\tokio_cpu_demo)
└── tokio v0.2.22
├── tokio-macros v0.2.5

Platform
Manifests on:

Windows 10 64-bit
Ubuntu 20.04 64-bit (note you'll have to change the hard-coded port 80 to something else on Linux.)

Description

A basic TCP-based web server, using tokio, when serving many requests, causes one or more CPUs to start running at 100% and never stop running.

The repro might be hard, sorry about that. To repro, run the program below, and spam GET requests at it (holding down F5 on a web browser usually triggered it within a few seconds, or you can write a script that spams GETs). In Task Manager, you can see the CPU usage of the program jump to 100/n % (where n is the number of CPUs). Sometimes it's k * 100/n % (where k is 1-4), but usually 100/n. Once the problem has been triggered, the CPU usage will never drop, even when you stop sending requests, and the program should be idle.

When I ran this program with a custom single-threaded tokio runtime, the program completely hung when the problem was triggered.

When I ran the program in Visual Studio with profiling turned on, after the problem was triggered, the profiler showed (in descending order of Total CPU):

tokio_cpu_demo::parse_method::{{closure}}
_ZN3std9panicking3try17he8a6adfc78550a4bE
_ZN71_$LT$tokio..time..instant..Instant$u20$as$u20$core..cmp..PartialOrd$GT$2gt17h75b077d7ec869541E
tokio_cpu_demo::parse_request_header::{{closure}}
... etc ...

/*
CARGO.TOML:


[package]
name    = "tokio_cpu_demo"
version = "0.1.0"
authors = ["Ross"]
edition = "2018"

[dependencies]
tokio           = { version = "0.2.22", features = ["full"] }
*/


use tokio::io::{AsyncBufRead, BufReader, AsyncBufReadExt};
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
use std::error::Error;
use std::marker::Unpin;
use std::net::SocketAddr;

// In a real web server, we would parse the "GET" line here.
async fn parse_method(r: &mut (dyn AsyncBufRead + Unpin + Send)) {
    let mut line = String::new();
    loop {
        let _ = r.read_line(&mut line).await.unwrap();
        if !line.trim().is_empty() {
            break;
        }
    }
}

// In a real web server, we would parse the header here.
async fn parse_request_header(r: &mut (dyn AsyncBufRead + Unpin + Send)) {
    parse_method(r).await;
    loop {
        let mut line = String::new();
        r.read_line(&mut line).await.unwrap();
        if line.trim().is_empty() {
            break;
        }
    }
}

fn error_404() -> String {
    format!(
        "{}\r\n{}\r\n{}\r\n\r\n{}\r\n",
        "HTTP/1.1 404 Not Found",
        "Connection: Close",
        "Content-Type:text/html",
        "<html><head><title>404</title></head><body>Nothing here</body></html>"
    )
}

async fn handle_get<W: AsyncWrite + Unpin + Send>(mut w: W) {
    let response = error_404();
    w.write_all(response.as_bytes()).await.unwrap();
}

async fn route_request<R>(mut r: R, w: &mut (dyn AsyncWrite + Unpin + Send))
where
    R: AsyncBufRead + Unpin + Send,
{
    let _header = parse_request_header(&mut r).await;
    handle_get(w).await;
}

async fn process_http_request(mut s: TcpStream) {
    dbg!();
    let (r, mut w) = s.split();
    route_request(BufReader::new(r), &mut w).await;
}

async fn main_loop(listen_url: SocketAddr) -> Result<(), Box<dyn Error>> {
    let mut listener = TcpListener::bind(&listen_url).await?;

    loop {
        let (socket, _) = listener.accept().await?;
        tokio::spawn(async move {
            process_http_request(socket).await;
        });
    }
}

#[tokio::main]
async fn main() {
    main_loop("0.0.0.0:80".parse().unwrap()).await.unwrap();
}

Answered by Darksonn

Oct 6, 2020

Have you verified that it isn't stuck in an infinite loop in parse_method? That will happen if the stream is closed before that function returns.

View full answer

Darksonn · 2020-10-06T11:47:00Z

Darksonn
Oct 6, 2020
Maintainer

Have you verified that it isn't stuck in an infinite loop in parse_method? That will happen if the stream is closed before that function returns.

1 reply

rosscanning Oct 6, 2020
Author

Ah, you're absolutely right. My mistake, sorry.

Thanks for the help!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tokio runtime: CPUs get maxed when serving many requests #2915

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 1 comment 1 reply

{{title}}

{{title}}

Select a reply

Tokio runtime: CPUs get maxed when serving many requests #2915

rosscanning Oct 6, 2020

Replies: 1 comment · 1 reply

Darksonn Oct 6, 2020 Maintainer

rosscanning Oct 6, 2020 Author

rosscanning
Oct 6, 2020

Replies: 1 comment 1 reply

Darksonn
Oct 6, 2020
Maintainer

rosscanning Oct 6, 2020
Author