✨ Implement MessageWriter trait

[christeefy]

Migrate lib.rs's PipesFileMessageWriter implementation into MessageWriter and MessageWriterChannel traits, following the PipesMessageWriter and PipesMessageWriterChannel Python base classes.

Note

MessageWriter::open differs from the Python's PipesMessageWriter.open implementation. Rust has no simple way to yield (afaik). Python's implementation yields so that it can perform cleanup. Instead, in the Rust implementation, we delegate the cleanup responsibility to the Drop trait — loosely analogous to a C destructor.

This is a simplified implementation that ignores lower-level LogWriter and LogWriterChannel abstractions in the Python implementation, to prevent this PR from getting larger than it already is.

This PR ended up using two advanced Rust functionalities, so I've included write-ups and inline documentation for posterity.

1. MessageWriter and MessageWriterChannel as associated types

After a few attempts of implementing it in Rust (and with help from ChatGPT), I've settled on using Associated Types:

trait MessageWriterChannel;

trait MessageWriter {
    type Channel: MessageWriterChannel;
    ...
};

The original reason for me doing this is to limit the "blast radius" caused by the PipesDefaultMessageWriter struct.
That struct returns different types of MessageWriterChannels, decided at runtime (see Python implementation). In Rust, every type must be known at compile-time (allowing for static dispatch and optimizations), otherwise they need to be wrapped in Box (a smart pointer) for dynamic dispatch which incurs a runtime cost.

Crucially, PipesDefaultMessageWriter's implementation would alter MessageWriter's trait signature:

// Initial, non-associated type implementation
trait MessageWriter {
-    fn open(&self, params: Map<String, Value>) -> impl MessageWriterChannel;
+    fn open(&self, params: Map<String, Value>) -> Box<dyn MessageWriterChannel>;
}

forcing all future (community) MessageWriter implementations to incur this type complexity (needing to understand Box and dynamic dispatch) & performance penalty.

By using associated types, each concrete MessageWriter implementation is free to return whatever Channel it needs without affecting other implementations, so long as that Channel implements the MessageWriterChannel type:

struct DefaultWriter { ... };
impl MessageWriter for DefaultWriter {
    type Channel = Box<dyn MessageWriterChannel>;
    ...
}

// Other implementations' `Channel`s don't have to be boxed
struct GCSWriter { ...};
impl MessageWriter for GCSWriter {
    type Channel = GCSMessageChannel;  // Statically-dispatched and can be optimized by the compiler
    ...
}

Additional Benefits

Using associated types also led to other unexpected benefits:

• The two given traits are "linked" — any MessageWriter implementation must have a corresponding and specific MessageWriterChannel implementation. In this PR, PipesDefaultMessageWriter must be used with Box<dyn MessageWriterChannel> and Box<dyn MessageWriterChannel> only, even if other MessageWriterChannel implementations exist¹.
• This means that the compiler will ensure that anytime we use PipesContext with a specific MessageWriter, the correct and corresponding MessageWriterChannel is used:

pub struct PipesContext<W>
where
    W: MessageWriter,
{
    data: PipesContextData,
    // Compiler enforces the correct `Channel` associated with a concrete `MessageWriter` at compile time
    message_channel: W::Channel, 
}

• This pattern can be reused when implementing LogWriter and LogWriterChannel in the future

2. Sealing trait methods

The PipesMessageWriter.get_opened_payload abstract method in Python is denoted with @final, preventing overrides. Stable Rust doesn't have a #[final] attribute yet, and so we'll have to manually seal that method.

This is done using a private::Token that's only accessible within that module. Callers outside that module cannot access this dummy struct, and therefore cannot override its implementation!

However, external callers also cannot call the function without private::Token, so a public function is provided.

// message_writer.rs
mod private { 
    pub struct Token;
}

struct DefaultWriter { ... };
impl MessageWriter for DefaultWriter {
    fn get_opened_payload(&self, _: private::Token);
}

/// Public accessor to the sealed method
pub fn get_opened_payload(writer: &impl MessageWriter) -> HashMap<String, Option<Value>> {
    writer.get_opened_payload(private::Token)
}

User-facing documentation is also included in the implementation's documentation!

TODOs

• Unit tests
• Error handling (can be done in a subsequent PR) to not make this PR any larger

Footnotes

1. For a generalized version of this, there's Generic Associated Types (GAT) ↩

[christeefy]

Fixes #13

[cmpadden]

Thank you @christeefy -- very appreciative that you took the time to outline the decision in the PR.

I'm going to merge this for now, as I plan to move this code base to the community-integration repo soon.

Side note, I plan on persisting the Git history in migration, so attribution will not be lost.