Seeking Initial Community Review of dCBOR

[shannona]

Announcing Our dCBOR Libraries

This week, we released our Rust and Swift libraries for dCBOR. These are CBOR codecs that focus on writing and parsing "deterministic" CBOR per §4.2 of RFC-8949, to enable the encoding and decoding of a variety of data types. It does not support parts of the spec forbidden by deterministic CBOR (such as indefinite length arrays and maps).

• dCBOR Codec for Rust
• dCBOR Codec for Swift

We've also built a dCBOR CLI atop the Rust library, to demonstrate its usage for parsing and validating dCBOR.

What & Why dCBOR?

dCBOR is "deterministically-encoded CBOR".

CBOR itself is the serialization format has a number of advantages, including conciseness, full extensibility, self description, and ease of use in constrained and streaming environment. Our "Why CBOR?" article talks about many of these advantages in depth and how they line up with many of our Gordian Principles and thus why we used them for Gordian Envelope.

dCBOR is then a variant of CBOR defined in the RFC that has specific encoding and ordering requirements to ensure the same CBOR data is always encoded ("deterministically") in the same way. It was critical to our development of Gordian Envelope because the Envelopes are built up as Merkle Trees of hashes, and we need those hashes to always be the same, which requires the data to always to be arranged the same. We expect there are many other use cases where full determinism in data storage is crucial, and our dCBOR Codecs are available for those usages.

Supporting Our Community Review

We are beginning an Initial Community Review of our new Codec reference libraries.

This is a very important part of our development path because Blockchain Commons is all about bringing the community together, to define their requirements and needs so that we can work together to produce interoperable specifications that are usable by everyone.

We're asking you as community members to look at our new libraries, to see how they meet your needs, and to advise us on how they could do so better. We'd also love your help to review actual usage of the libraries. Do they work for you? Are they intuitive? Did you run into any bugs or problems? What do you think of the API? How about the Rust idioms (if you use Rust)? Are there code security tools that you'd like to see added?

Finally, you can also help us by providing us with your own CBOR documents that we can test against or use in our docs, such as a valid mDL or COSE.

Looking toward the Future

This is just the first step.

We hope to expand the dCBOR project to include other languages. One of our sponsors is contributing Kotlin bindings and we're seeking others. We'd also like someone to do a Python port and would welcome other ports.

Finally, we are coordinating a special budget from our sponsors to do an independent third-party audit of the code. Please help us by becoming a sponsor if you're not already, as this is important code to get right!

[cyberphone]

I have reviewed the part of dCBOR that deals with number serialization. Although there is nothing fundamentally wrong with dCBOR's take on the matter, it departs from an informal "de-facto" standard. In the table in
https://www.rfc-editor.org/rfc/rfc8949.html#section-appendix.a
you can find that the value -4.0 is supposed to serialize as 0xf9c400. However, in the current version of dCBOR it would serialize as 0x23. This table is (unsurprisingly) also the foundation for the encoding used by the CBOR playground (https://cbor.me/) maintained by the principal CBOR RFC author, Carsten Bormann.

IMO, dCBOR would from an adoption point of view gain by sticking to the de-facto standard. The downside of non-optimal representation is not a showstopper. Regarding determinism, the only hard requirement is that you get repeatable results which both solutions provide.

I understand that one of the motives behind the current design, was to relieve developers from number serialization details. This is probably not possible using the de-facto standard. However, a properly designed Gordian Envelope "builder" API should be able shielding user level developers from all serialization issues, like https://www.bouncycastle.org/docs/pkixdocs1.5on/org/bouncycastle/cert/X509v3CertificateBuilder.html.

[cyberphone]

Regarding APIs, I have in my own take on deterministic CBOR, built CBOR wrappers that does the actual serialization.

byte[] cbor = new CBORMap()
                  .setObject(new CBORInteger(1), new CBORString("PI"))
                  .setObject(new CBORInteger(2),new CBORFloatingPoint(3.14)).encode()

This should generate the following CBOR, here in diagnostic notation:

{
  1: "PI",
  2: 3.14
}