Need for feedback and road to hydra 1.0.0 stabilization!

[dtzxporter]

Hi All,

We're looking for feedback on the current state of the hydra framework. Hydra is deployed at 4 different companies that I know of operating at a large scale (prior to being open-sourced).

We want to stabilize the API as soon as possible, so any feedback would be appreciated!

[hampuskraft]

Hey! I love your work on Hydra. I'm migrating four Elixir services to Rust & Hydra and am very happy so far! One thing that I'm still trying to figure out is exposing an HTTP or gRPC service alongside Hydra (that part I can get working by spawning a process) that can also communicate with other processes on the node (I'm getting "cannot access a task-local storage value without setting it first" upon trying to do that). My only feedback thus far is to add an official example for this pattern — or some pointers in the right direction; it might be easier than I think. Thanks!

[dtzxporter]

Thanks for the feedback! Yeah, I need to document the cases where it's ok to cross a 'task' -> 'process' boundary for certain actions. I'm also trying to get Tokio to stabilize their TaskId which would prevent the need to wrap processes in a TaskLocal. I understand this is a common pattern though and will work to improve/provide a sensible default!

Also, I need to improve that error message, haha!

[seivan]

I have the same issue. I was hoping to just create a GenServer that stores the webserver and all its related bootstrap stuff.
Like if you're basically launching https://loco.rs inside a GenServer and have its controllers and other stuff send messages to other processes.

Obviously it wouldn't have the benefit of Elixirs fault tolerance where every request is its own process, but at least it will set up a nice structure.

[cpud36]

Hi, I love your work on this crate. I really like the dynamic send/recv part.

There is quite a few of suggestions, I hope you will find them valuable.

Exposing free functions

Have you considered exposing Process functions as e.g. hydra::send, instead of hydra::Process::send?
It should be cleaner and more idiomatic.

Recv Multiple Types

Currently you can only recv a single type of messages. This makes it difficult to handle multiple tasks in single actor(because your users have to know all messages you are able to handle atm).

I suggest exposing roughly the following function:

fn raw_select<T>(selector: FnMut(MessageSlot<'_>) -> Option<T>) -> T;

let value = raw_select(|slot| {
  if let Some(item) = slot.get::<MyMsg>() {
    // item has type TypedMsgSlot<'_, MyMsg>
    if item.x > 0 {
        return Some(item.take());
    }
  }
  None
})

And also exposing an hydra::receive! macro as a user facing api.

Handling system messages

Currently the user is forced to handle system messages on every receive. On the contrary in erlang, are either handled automatically(e.g. exit, unless trapped), or are just added to the mailbox, and it is the user to decide whether to handle system messages.

Although it might seem like a reliability hazard, it works quite well in practice.

Type confusion in deserialisation

Currently, if you send a message A, and recv a message B it might succeed, because the message is serialised as MessagePack, and does not have any type tag.

#[derive(Clone, Serialize, Deserialize)]
struct CheckDbWal {
  db_id: u32,
  last_gc: Instant,
}

#[derive(Clone, Serialize, Deserialize)]
struct DropDb {
  db_id: u32,
}

// first local replica will check db wal 
// all other replicas might drop db...
Process::send(replicas, CheckDbWal { .. })

Incorporating sender pid

In erlang, recv returns not only the message, but also a sender pid. This makes feasible to program without a GenServer. Have you considered incorporating this into core protocol?

Error handling differences with erlang

In erlang, send is always successful, and recv just ignores unknown messages.
Is there any rationale behind your decision?

Require Clone for messages

Essentially any Encode+Decode is effectively Clone, so it should be reasonable to require this explicitly.

[dtzxporter]

Thanks for the feedback, I've addressed the remote protocol issue (missed the config option for msgpack...) I will read/follow up in a few weeks (on holiday) to the rest!