Use LocalPoolHandle to use multiple cpu core

[yongman]

I use LocalSet to spawn !Send future, and the performance is bad, because just use one cpu core used with LocalSet spawn.
And I learn the example of the async_tcp_server.

I want to know the pattern how to use LocalPoolHandle to balance the accepted connections to workers in async_tcp_server.

LocalPoolHandle doesn't have run_until method or similar, and I can not do await in loop in run_server which result loop block.

I am a fresher to rust and mlua, looking forward to any suggestions. :-)

[khvzak]

A single Lua instance can utilize only a single core so a common practice is use a Lua instance per core and loadbalance work between VMs.
If speak about accepting incoming tcp connections and balancing them between workers you can use approach described in https://docs.rs/tokio/latest/tokio/task/struct.LocalSet.html#use-inside-tokiospawn

1. create a set of workers that uses mpsc channel to accept tasks
2. create a main listener and start listening for connections
3. send new connection to a next worker

Given that Lua perfectly integrates with Rust async, one lua VM can process thousands connections simultaneously.

[khvzak]

This may result that I have to create thousands of thread workers at start？

Why? You can handle thousands of connections in one thread / single Lua VM.

[khvzak]

In particular you need to use Local::spawn_local to handle a new connection on desired worker/thread and you can make as many spawn_local as you want (no need to await them).
It uses cooperative multitasking and allow to simultaneously process many connections in one OS thread.

[yongman]

Thanks for your quick reply.

Oh I see, await is not needed after spawn_local.
I will dive into it later.

[yongman]

In particular you need to use Local::spawn_local to handle a new connection on desired worker/thread and you can make as many spawn_local as you want (no need to await them).

struct LocalSpawner {
   send: mpsc::UnboundedSender<Handler>,
}

impl LocalSpawner {
    pub fn new() -> Self {
        let (send, mut recv) = mpsc::unbounded_channel::<Handler>();
        
        std::thread::spawn(move || {
            let local = LocalSet::new();

            let rt = Builder::new_current_thread()
            .enable_all()
            .build()
            .unwrap();

            local.spawn_local(async move {
                while let Some(mut handler) = recv.recv().await {
                        // Process the connection. If an error is encountered, log it.
                        // handle.run will be loop runing inside until connection close
                        if let Err(err) = handler.run().await {
                            println!("Connection Error: {:?}", &err);
                            error!(cause = ?err, "connection error");
                        }
                }
                // If the while loop returns, then all the LocalSpawner
                // objects have have been dropped.
            });

            rt.block_on(local);
        });

        Self {
            send,
        }
    }

    pub fn spawn(&self, handler: Handler) {
        self.send.send(handler).expect("Thread with LocalSet has shut down.");
    }
}

I try to implement spawn_local as above, the main thread will be blocked when I call rt.block_on(local) and only one connection
can be handled.
If I use async closure to execute local.await, the std::thread::spawn warned me that async closure must be Send too.
I want to know how can I do await the local to run tasks. :-(

[yongman]

At last I figure it out using LocalPoolHandle. Do not need to await in JoinHandle as you suggested.
Thanks for your patient reply. ❤️