Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP add backend messages serialization and frontend messages deserialization #946

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

prx0
Copy link

@prx0 prx0 commented Aug 26, 2022

PR related to #942

The goal is to allow serialization of backend message into bytes and deserialization of frontend message into rust object that we can handle.
It could be useful for making a proxy or a fake postgresql server. (honeypot)

I've still lot of work to do here. My approach for backend message serialization is to implement TryFrom for Message in order to convert them into bytes.
I'm open to suggestions if you know a better way to do this of course.

Postgresql message flow documentation: https://www.postgresql.org/docs/current/protocol-message-formats.html

Thank you!

@prx0
Copy link
Author

prx0 commented Sep 11, 2022

Hi Max, I actually wanted to implement the same thing as I was inspired by https://github.com/jackc/pgproto3. You can get a lot of pointers for parsing the messages there

Hello!

Thank you. I use already pgproto3 in my company but I still want to continue this PR because I would like to work with Rust.

I'll check out the source code of pgproto3 to get some help.

Backend serialization is quite easy. But frontend deserialization is not.

I need some motivations and time, but I promess I'll finish this PR. ^^'

@zain-kabani
Copy link

Hi I've been working on a project to make creating postgres proxies easier in rust. I've just gotten started on it and welcome any contributions to it! https://github.com/zain-kabani/postgres-proto-rs

@prx0
Copy link
Author

prx0 commented Sep 15, 2022

@zain-kabani that's great! I noticed that you've started to implement deserialization for frontend messages. That's what I try to do here. (also be careful you didn't specify any license in your project)

I don't know if it's a good idea to create another crate to add postgres protocol parsing because the rust-postgres crate made by @sfackler do this already. rust-postgres is a dependency of the postgres crate which is massively used. It would be great to add deserialization and serialization into rust-postgres in order to add functionnalities for existing projects based on this crate.

I think it would be awesome to create a framework for making progresql proxies with rust-postgres as dependency. pgcat is another inspiring project.
https://github.com/levkk/pgcat
Not a framework but a tool to manage sharding avec load balancing for postgres databases.

EDIT: your frontend messages are mainly struct with a ByteMut attribute called message_byte. That's a simple approach I like. In my case I would like to implement something more like the structs from jackc/pgproto3.
The idea is to be able to modify each parameter of a frontend message.

Example for a BindBody

#[non_exaustive]
pub enum Message {
    Bind(BindBody),
    Close(CloseBody),
    CopyFail(CopyFailBody),
    Descibe(DescribeBody),
    Execute(ExecuteBody),
    Flush(FlushBody),
    FunctionCall(FunctionCall),
    GSSENCRequest(GSSENCRequestBody),
    GSSResponse(GSSResponseBody),
    Parse(ParseBody),
    PasswordMessage(PasswordMessageBody),
    Query(QueryBody),
    SASLInitialResponse(SASLInitialResponseBody),
    SASLResponse(SASLResponseBody),
    SSLRequest(SSLRequestBody),
    StartupMessage(StartupMessageBody),
    Sync(SyncBody),
    Terminate(TerminateBody),
}

// highly based on https://github.com/jackc/pgproto3/blob/master/bind.go
pub struct BindBody {
    pub destination_portal: String,
    pub prepared_statement: String,
    pub parameter_format_codes: Vec<u16>,
    pub parameters: Vec<Vec<Bytes>>,
    pub result_format_codes: Vec<u16>,
}

I don't know if it's a good idea to go into such details than using a simple attribute which do the job.

@zain-kabani
Copy link

Hi Max, yeah I considered adding to this repo too but didn't want to rock the boat too much here and decided to just create a separate project.
Fun fact I'm actually an active contributor to the pgcat project, and built this with pgcat in mind.

Yes those are there temporarily as I still have to implement the logic of encoding/decoding them. (contributions are welcome!)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants