Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: serialize raw results as arrays instead of object #4939

Merged
merged 11 commits into from
Jul 12, 2024

Conversation

Weakky
Copy link
Contributor

@Weakky Weakky commented Jul 1, 2024

Comment on lines 193 to 226
#[derive(Debug, Copy, Clone, PartialEq, Serialize)]
enum SerializedValueType {
#[serde(rename = "int")]
Int32,
#[serde(rename = "bigint")]
Int64,
#[serde(rename = "float")]
Float,
#[serde(rename = "double")]
Double,
#[serde(rename = "string")]
Text,
#[serde(rename = "enum")]
Enum,
#[serde(rename = "bytes")]
Bytes,
#[serde(rename = "bool")]
Boolean,
#[serde(rename = "char")]
Char,
#[serde(rename = "decimal")]
Numeric,
#[serde(rename = "json")]
Json,
#[serde(rename = "xml")]
Xml,
#[serde(rename = "uuid")]
Uuid,
#[serde(rename = "datetime")]
DateTime,
#[serde(rename = "date")]
Date,
#[serde(rename = "time")]
Time,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried to clean that up using this macro:

macro_rules! impl_value_type {
    (
        $($variant:ident => ($mapped_variant:ident, $rename:expr)),*;
        $($array_variant:ident[] => ($mapped_array_variant:ident, $array_rename:expr)),*;
        $(($custom_variant:ident, $custom_rename:expr)),*
    ) => {
        #[derive(Debug, Clone, Serialize)]
        enum SerializedValueType {
            $(
                #[serde(rename = $rename)]
                $mapped_variant,
            )*
            $(
                #[serde(rename = $array_rename)]
                $mapped_array_variant,
            )*
            $(
                #[serde(rename = $custom_rename)]
                $custom_variant,
            )*
        }

        impl<'a> From<Value<'a>> for SerializedValueType {
            fn from(value: Value<'a>) -> Self {
                match &value.typed {
                    $(
                        ValueType::$variant(..) => SerializedValueType::$mapped_variant,
                    )*
                    ValueType::Array(values) => {
                        match values {
                            Some(values) if values.is_empty() => SerializedValueType::UnknownArray,
                            None => SerializedValueType::UnknownArray,
                            Some(values) => {
                                match &values[0].typed {
                                    $(
                                        ValueType::$array_variant(..) => SerializedValueType::$mapped_array_variant,
                                    )*
                                    ValueType::Array(_) | ValueType::EnumArray(_, _) => unreachable!()
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

impl_value_type! {
    Int32 => (Int32, "int"),
    Int64 => (Int64, "bigint"),
    Float => (Float, "float"),
    Double => (Double, "double"),
    Text => (String, "string"),
    Enum => (Enum, "enum"),
    EnumArray => (TextArray, "string-array"),
    Bytes => (Bytes, "bytes"),
    Boolean => (Boolean, "bool"),
    Char => (Char, "char"),
    Numeric => (Numeric, "decimal"),
    Json => (Json, "json"),
    Xml => (Xml, "xml"),
    Uuid => (Uuid, "uuid"),
    DateTime => ( DateTime, "datetime"),
    Date => (Date, "date"),
    Time => (Time, "time");

    Int32[] => (Int32Array, "int-array"),
    Int64[] => (Int64Array, "int64-array"),
    Float[] => (FloatArray, "float-array"),
    Double[] => (DoubleArray, "double-array"),
    Text[] => (TextArray, "string-array"),
    Enum[] => (TextArray, "string-array"),
    Bytes[] => (BytesArray, "bytes-array"),
    Boolean[] => (BooleanArray, "boolean-array"),
    Char[] => (CharArray, "char-array"),
    Numeric[] => (NumericArray, "numeric-array"),
    Json[] => (JsonArray, "json-array"),
    Xml[] => (XmlArray, "xml-array"),
    Uuid[] => (UuidArray, "uuid-array"),
    DateTime[] => (DateTimeArray, "datetime-array"),
    Date[] => (DateArray, "date-array"),
    Time[] => (TimeArray, "time-array");

    (Unknown, "unknown"),
    (UnknownArray, "unknown-array")
}

Didn't work well because EnumArray maps to TextArray, but so does Text[]. Also, I need to map Array[] and EnumArray[] to unreachable!(). Lemme know if you find a better way.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the problem can be solved through procedural macro, because you can de-duplicate enum variants. This, however, is a bit more effort. I will leave it up to you to decide which approach to go with, I don't feel strongly about it

Copy link

codspeed-hq bot commented Jul 1, 2024

CodSpeed Performance Report

Merging #4939 will not alter performance

Comparing perf/query-raw-serialization (8faf64c) with main (9618390)

Summary

✅ 11 untouched benchmarks

Copy link
Contributor

github-actions bot commented Jul 1, 2024

WASM Query Engine file Size

Engine This PR Base branch Diff
Postgres 2.053MiB 2.044MiB 9.299KiB
Postgres (gzip) 820.248KiB 814.575KiB 5.673KiB
Mysql 2.022MiB 2.014MiB 9.043KiB
Mysql (gzip) 806.667KiB 801.013KiB 5.655KiB
Sqlite 1.922MiB 1.915MiB 7.749KiB
Sqlite (gzip) 768.549KiB 762.983KiB 5.567KiB

@Weakky Weakky added this to the 5.17.0 milestone Jul 1, 2024
@Weakky Weakky marked this pull request as ready for review July 1, 2024 14:05
@Weakky Weakky requested a review from a team as a code owner July 1, 2024 14:05
@Weakky Weakky requested review from laplab and removed request for a team July 1, 2024 14:05
Comment on lines 193 to 226
#[derive(Debug, Copy, Clone, PartialEq, Serialize)]
enum SerializedValueType {
#[serde(rename = "int")]
Int32,
#[serde(rename = "bigint")]
Int64,
#[serde(rename = "float")]
Float,
#[serde(rename = "double")]
Double,
#[serde(rename = "string")]
Text,
#[serde(rename = "enum")]
Enum,
#[serde(rename = "bytes")]
Bytes,
#[serde(rename = "bool")]
Boolean,
#[serde(rename = "char")]
Char,
#[serde(rename = "decimal")]
Numeric,
#[serde(rename = "json")]
Json,
#[serde(rename = "xml")]
Xml,
#[serde(rename = "uuid")]
Uuid,
#[serde(rename = "datetime")]
DateTime,
#[serde(rename = "date")]
Date,
#[serde(rename = "time")]
Time,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the problem can be solved through procedural macro, because you can de-duplicate enum variants. This, however, is a bit more effort. I will leave it up to you to decide which approach to go with, I don't feel strongly about it

@CLAassistant
Copy link

CLAassistant commented Jul 11, 2024

CLA assistant check
All committers have signed the CLA.

@Weakky Weakky merged commit c4fe130 into main Jul 12, 2024
208 checks passed
@Weakky Weakky deleted the perf/query-raw-serialization branch July 12, 2024 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants