Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce more serde utils #92

Closed
wants to merge 2 commits into from

Conversation

tcharding
Copy link
Member

Copy the serde_utils modules for ser/deser of BTreeMaps. Currently includes the hex_bytes module even though it would seem we don't need it but I couldn't get it to work with the current functions. Its private as opposed to the original where it was public.

@tcharding
Copy link
Member Author

If this lands we can use this to assist the crate smashing effort in rust-bitcoin. The new crates need these utils.

@tcharding tcharding force-pushed the 06-20-serde-utils branch 4 times, most recently from 4dc955c to 9866e27 Compare June 20, 2024 05:07
@apoelstra
Copy link
Member

To clarify -- you copied this from rust-bitcoin? Or from std?

@apoelstra
Copy link
Member

Ah, it's from rust-bitcoin. Cool.

@apoelstra
Copy link
Member

@tcharding can you update Cargo.toml to turn on the derive feature of serde even when it's not a dev-dependency?

I'm not sure why cargo seems to be OK with the current PR but it's technically incorrect and my local non-cargo tooling is breaking on it.

We derive `serde` traits so we need the `derive` feature to be enabled
in all builds not just dev builds.

Presumably this bug has not shown up because anytime `serde` is enabled
some other crate in the stack enabled `derive`.
Copy the `serde_utils` modules for ser/deser of `BTreeMap`s. Currently
includes the `hex_bytes` module even though it would seem we don't need
it but I couldn't get it to work with the current functions. Its private
as opposed to the original where it was public.
@tcharding tcharding force-pushed the 06-20-serde-utils branch from 7d2091d to 36cf5d7 Compare June 23, 2024 23:57
@tcharding
Copy link
Member Author

Added an additional patch up front.

Copy link
Collaborator

@Kixunil Kixunil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the first commit is just plain wrong (see below) and second has questionable usefulness.

Side note: I never intended for hex-conservative to be a dumping ground for random hex-related shit. I want it to be a nice idiomatic crate. Adding stuff that's only used in one crate until someone else needs it is not helpful to achieve this goal.

@@ -27,11 +27,10 @@ alloc = []
[dependencies]
arrayvec = { version = "0.7.2", default-features = false }
core2 = { version = "0.3.2", default-features = false, optional = true }
serde = { version = "1.0", default-features = false, optional = true }
serde = { version = "1.0", default-features = false, features = ["derive"], optional = true }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, where do you see any use of derive? I only see it in examples.

@@ -106,3 +106,304 @@ where
d.deserialize_map(HexVisitor(PhantomData))
}
}

pub(crate) struct SerializeBytesAsHex<'a>(pub(crate) &'a [u8]);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this no longer needs pub(crate).

d.deserialize_str(Visitor(core::marker::PhantomData))
}
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These btree map things are super-specific and I don't think they'll be shared by many crates. Even in bitcoin we use it for PSBT which has weird serialization. I'd rather remove them entirely rather than put them here.

@Kixunil
Copy link
Collaborator

Kixunil commented Jun 30, 2024

my local non-cargo tooling is breaking on it.

Oh, this PR causes it, it wasn't a problem before. Having it in two commits is confusing.

So if this is accepted I'd prefer to squash the commits since they don't make sense separated. That being said, adding another dependency for this is annoying, it'd be nice trying to impl it manually since it just uses serialize_with anyway.

@tcharding tcharding closed this Jun 30, 2024
@apoelstra
Copy link
Member

Side note: I never intended for hex-conservative to be a dumping ground for random hex-related shit

Should we start bitcoin-hex then with hex-related shit that we need for rust-bitcoin?

@Kixunil
Copy link
Collaborator

Kixunil commented Jul 1, 2024

This particular one is only needed for PSBT. If we need it for multiple things and multiple crates then that would be a strong indication that putting it into hex-conservative is actually worth it. (But we could let it sit for a bit in internals.)

@apoelstra
Copy link
Member

@tcharding is it only PSBT?

Actually, if it's really only PSBT then we can maybe get away with dropping the serde serialization entirely and just serializing as strings... I really dislike breaking serde de/serialization but this one is really ad-hoc.

@Kixunil
Copy link
Collaborator

Kixunil commented Jul 1, 2024

Exactly. Serialize as base64.

@tcharding
Copy link
Member Author

Used also in witness, I'll investigate if we can remove that.

bitcoin/src/blockdata/witness.rs:476: seq.serialize_element(&crate::serde_utils::SerializeBytesAsHex(elem))?;

@Kixunil
Copy link
Collaborator

Kixunil commented Jul 2, 2024

Damn, I had a huge brain fart yesterday forgetting there's also SerialzieBytesAsHex. That one is generally useful I believe and should be added. Sorry about the noise.

@tcharding
Copy link
Member Author

For posterity, this was closed in favour of #96

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants