SQLx from a function?

[ElhamAryanpur]

Hey there! I wanted to call to an SQLx database from within an mlua async function.

Biggest issue I came across was that I think mlua uses futures-utils and tokio based sqlx straight up refused to work, and async-std based sqlx freezes the compiler, and doesn't return anything.

Is there any way to run a tokio or async-std based library in mlua async functions?

A large amount of my codebase depends on lua and this is a hard need...

[ElhamAryanpur]

Solution for those seeking is to use Tokio's handle. Possible through a static once_cell. And then spawn an async thread to do the task you want.

this stackoverflow thread can be helpful: https://stackoverflow.com/questions/61292425/how-to-run-an-asynchronous-task-from-a-non-main-thread-in-tokio

[khvzak]

Do you use module mode?
Otherwise unlikely you need to do anything else apart from having

#[tokio::main]
async fn main() {}

[khvzak]

There should be no issues in any version.

This is an example how to run multiple tokio tasks concurrently in v0.9:

use mlua::{Lua, Result};
use std::rc::Rc;
use std::time::Duration;

async fn sleep(_: &Lua, ms: u64) -> Result<()> {
    tokio::time::sleep(Duration::from_millis(ms)).await;
    println!("Slept for {ms} ms");
    Ok(())
}

#[tokio::main]
async fn main() {
    let local = tokio::task::LocalSet::new();
    local
        .run_until(async {
            let lua = Rc::new(Lua::new());

            lua.globals()
                .set("sleep", lua.create_async_function(sleep).unwrap())
                .unwrap();

            // Run few tasks concurrently
            for i in 1..10 {
                let lua = lua.clone();
                tokio::task::spawn_local(async move {
                    lua.load(&format!("sleep({i})")).exec_async().await.unwrap();
                });
            }
        })
        .await;
    local.await;
}

This is example for main branch with send feature flag:

use mlua::{Lua, Result};
use std::time::Duration;

async fn sleep(_: Lua, ms: u64) -> Result<()> {
    tokio::time::sleep(Duration::from_millis(ms)).await;
    println!("Slept for {ms} ms");
    Ok(())
}

#[tokio::main]
async fn main() {
    let lua = Lua::new();

    lua.globals()
        .set("sleep", lua.create_async_function(sleep).unwrap())
        .unwrap();

    let mut tasks = tokio::task::JoinSet::new();
    for i in 1..10 {
        let lua = lua.clone();
        tasks.spawn(async move {
            lua.load(&format!("sleep({i})")).exec_async().await.unwrap();
        });
    }

    while let Some(_) = tasks.join_next().await {}
}

[khvzak]

@cheesycod Yes, Lua is single threaded and mlua holds an exclusive mutex to protect the vm. To run multiple blocking tasks in parallel you ideally need a pool of VMs. However you still should be able to concurrently execute futures even using a single threaded vm. The lock is obtained when poll() futures and released after.

[khvzak]

Also note that in non-send version there is no lock and the previous approach (with LocalSet) would work as well.

[ElhamAryanpur]

Interesting, I couldn't get that syntax to work at all with tokio-cron, perhaps the requirement for 'static in the signature was the issue:

impl Job {
    pub fn new<S, F, Fut>(cron: S, func: F) -> Self
        where
            F: Fn() -> Fut + Send + Sync + 'static,
            Fut: Future<Output=()> + Send + 'static,
            S: Into<String>

[ElhamAryanpur]

I think I'll keep the code I have with the thread for now, like you said ideally a pool of VM would be much better of an idea for parallel execution, so will try to get something like that up in the future. Except using tokio threads instead of std threads to get the async fixes per your suggestions