uuidformattester Tool

[daegalus]

@kyzer-davis @bradleypeabody

Are we still considering this? I know you guys just got off making 9562 a reality (big congratulations on that), but was curious if this was a thing we were still pursuing as it's something I'm very interested in.

[bradleypeabody]

I'm still open to the idea. Just not sure if I have the energy to argue with everyone about it, lol. Prior discussions had a lot of varying opinions.

In general I still think that an alternate text encoding that had the following properties could be standardized:

• Fits as many use cases as possible - and we'll want an exact list with things like DNS records, URL path, URL query string, literals in various languages and config files, etc. - and this can serve as a guide that we can compare different options to and weigh the tradeoffs for each one.
• Sorts the same as the binary encoding - this way it doesn't matter internally if systems store as text or binary, the sort order is always the same, greatly simplifying questions around database indexes, search algorithms, etc. Note that this rules out a number of common encodings, including Base 64.
• Backward compatible parsing - meaning existing UUID Parse() functions can unambiguously parse these new values, and existing code can be updated in a logical and deterministic way.
• Does not depend on length - in case UUID-like values longer than 128 bits become a thing in the future, we shouldn't lock this spec into a specific length.
• Acknowledgement that this is not the only way in the universe to represent UUIDs as text, and the fact that such a spec exists doesn't prevent people from encoding in whatever other weird and wonderful ways they want.
• I think we should pick one single alternate shorter encoding, rather than selecting multiple. But if we did have multiple, I think it's critical that they can be unambiguously differentiated from each other. Otherwise anyone trying to write a Parse() function is going to then have the problem that whatever system must know which text format was used, and the whole thing generally just becomes a big mess where anyone can encoding anything in any format, in which case why have a standard if it doesn't help simplify implementations.

My review of this earlier led to Crockford Base 32 (https://www.crockford.com/base32.html) as the encoding I was in support of. But there are a lot of tradeoffs and things to consider, and no matter what is chosen there are (understandably) bound to be some people pissed off because their particular favorite encoding was not selected or something that doesn't fit their most pressing use case. That's part of the fun and excitement of pushing something like this through.

[daegalus]

• Fits as many use cases as possible - and we'll want an exact list with things like DNS records, URL path, URL query string, literals in various languages and config files, etc. - and this can serve as a guide that we can compare different options to and weigh the tradeoffs for each one.

Agreed, this eliminates most things base64 and up (unless we use emoji, which have a url safe representation, but still). And there is no point in doing base16 or lower as then it would be equal or worse than current UUID

• Sorts the same as the binary encoding - this way it doesn't matter internally if systems store as text or binary, the sort order is always the same, greatly simplifying questions around database indexes, search algorithms, etc. Note that this rules out a number of common encodings, including Base 64.

This is doable for Base32 and Base58 as they allow custom alphabets, and the alphabet can be setup in a way that sorts equally.

• Backward compatible parsing - meaning existing UUID Parse() functions can unambiguously parse these new values, and existing code can be updated in a logical and deterministic way.

This is doable also, most major libraries (including mine for Dart) need to handle UUIDs without dashes. Which then just makes it so we handle different bases based on some information

• Does not depend on length - in case UUID-like values longer than 128 bits become a thing in the future, we shouldn't lock this spec into a specific length.

Also valid, and not hard to do.

• Acknowledgement that this is not the only way in the universe to represent UUIDs as text, and the fact that such a spec exists doesn't prevent people from encoding in whatever other weird and wonderful ways they want.

Sure, but one of the many use cases is the interoperability between systems, mostly for being parsable and such by many libraries and languages.

• I think we should pick one single alternate shorter encoding, rather than selecting multiple. But if we did have multiple, I think it's critical that they can be unambiguously differentiated from each other. Otherwise anyone trying to write a Parse() function is going to then have the problem that whatever system must know which text format was used, and the whole thing generally just becomes a big mess where anyone can encoding anything in any format, in which case why have a standard if it doesn't help simplify implementations.

Also agreed. 1 is fine, but then you will get arguments on which 1 we want. We can take a page from Crypto, where addresses have a prefix that is always constant for a chain.

In this case we can use 1 or 2 bytes to denote the base, and variant. Maybe 1 byte and use the bits inside for flags. Base is self explanatory. 32, 58, etc. variant can be used to denote alphabet differences. Like Base64URL vs Base64. We can use 6 bits for base, and 2 bits for variant. Or 5 and 3. Something like that. That's an implementation detail.

My review of this earlier led to Crockford Base 32 (https://www.crockford.com/base32.html) as the encoding I was in support of. But there are a lot of tradeoffs and things to consider, and no matter what is chosen there are (understandably) bound to be some people pissed off because their particular favorite encoding was not selected or something that doesn't fit their most pressing use case. That's part of the fun and excitement of pushing something like this through.

That's fair. It is where I think a flexible prefix can help. Especially for Base32 that has multiple alphabets (I also maintain the Base32 library for Dart, so I'm familiar)

I genuinely think Base58 with a carefully ordered alphabet could give us the best results. Gets it down to 22 characters. Maybe 23 if we have that prefix. Much nicer than the

Not to mention the default alphabet is numbers, uppercase, lowercase, so it should sort fine.

[daegalus]

I think I'm going to create a small page where I can show the various outputs and such so we can better visualize

[sergeyprokhorenko]

What do you think about standardizing UUID color highlighting (with/without DB index, for light/dark theme), graphical representation (identicons, emoticons), short hash representation?

[sergeyprokhorenko]

It is where I think a flexible prefix can help

I think that the special data type "UUID with short format output" is much better than the prefix. Using this data type eliminates the need to parse the UUID with the prefix every time.

The UUID data type in PostgreSQL can be used as an example.

[daegalus]

It is where I think a flexible prefix can help

I think that the special data type "UUID with short format output" is much better than the prefix. Using this data type eliminates the need to parse the UUID with the prefix every time.

The UUID data type in PostgreSQL can be used as an example.

That makes sense of we choose 1 way of doing it.

If we allow customization, how will we know its emoji, Base32, Base58, or Base62? Especially if we can't rely on the length of the string.

[daegalus]

What do you think about standardizing UUID color highlighting (with/without DB index, for light/dark theme), graphical representation (identicons, emoticons), short hash representation?

Ya, that sounds fine to me. But i feel those are for a different discussion/ticket.

[sergeyprokhorenko]

What do you think about standardizing UUID color highlighting (with/without DB index, for light/dark theme), graphical representation (identicons, emoticons), short hash representation?

Ya, that sounds fine to me. But i feel those are for a different discussion/ticket.

See #6

[daegalus]

I am working on the webpage, but this is an initial result below. It is sortable if you use a sortable UUID (v6 or v7) for B32(crockford), B58, and B62

v7:

v4:

[daegalus]

Here you go guys, have fun: https://uuidformattester.yuli.dev/

Let me know if you want more encodings (emoji?) or if there are issues.

[kyzer-davis]

Great discussions so far.

Sorry I am late to the party here but yes, this is still on my TODO.
I have been busy splitting the "UUID Long" items out into their own doc so we can keep this one laser focused on Alt Encodings.

Going to do a bit of cleanup to the issue tracker, files and README so pardon the commits.

[daegalus]

Just know i can build custom encodings, with any prefixes and such as needed. So if anyone had a request, let me know

[kyzer-davis]

@daegalus, can you check the new TRADEOFF.md doc and use the starting UUID Binary/Integer representations in your code to force the encoding output from that common starting point?

I would like somebody sanity check my own outputs that are in the table.

Here is what I did in the event one is incorrect:

• For base16 (I used the RFC)
• For all base 32/64 I used https://cryptii.com/pipes/ to encode the binary (options shown in the dropdown)
  • 
  • 
• For Base58 (BTC) I used https://www.dcode.fr/chiffre-base-58
  • 
• For checking my own base32/64 outputs (except URL Safe/Hex) I ran them through the two below to ensure the integer returned matched the original starting point.
  • https://www.dcode.fr/code-base-64
  • https://www.dcode.fr/code-base-32-crockford
  • https://www.dcode.fr/code-base-32

Edit: I kept padding to illustrate it. I updated the tradeoff doc with a note that, depending on the algo, padding may be disabled.

[daegalus]

@kyzer-davis I have added onload entries. 2 entries 1 for each base32 variant listed. You can go see them on the page. I also verified base62 with dcode.fr, and that matches.

Hopefully that helps validate the code.

[kyzer-davis]

Took me a second to understand but you meant this:
https://uuidformattester.yuli.dev/?onload=329800735698586629295641978511506172918

That looks great!

Could you add data zbase and geohash to onload? or just append them to the same base32 cell and put a little parenthesis behind it that says like:

Base32 (type)
Z0EMZBKXXG8X19V502GCJ7KBYR (Crockfords) V0EKVBJTTG8T19R502GCI7JBUO====== (Base32hex) <VALUE> (geohash) <Value> (zbase32)

Then if you can add Base64 to the table in a similar display to Base32 that would be awesome.

---

The only other items I saw in past discussions that I need to verify, from that common starting point, for the tradeoff table are below:

• Base32 NCName
• Base64 NCName
• Base 48
• Base 52
• Base 36

If you want to take a stab at the 5 remaining items for your script that will help sanity check my work (and give us some nice implementation feedback).

The NCNAME ones can go in the Base32/Base64 cells abut 36/48/52 likely need columns... which means your app may need to go fullwidth for the table content :)

[daegalus]

Ya, give me a few days. I just moved to a new country (14hr flight, jetlag, 2 young kids, etc). So I need a bit to recover

[daegalus]

I have updated the tool to include 36/48/52/64 and added improved formatting.

NCName is a bit trickier, so I will get to that a bit later, since it has extra algorithm changes.