Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REP-2011] Evolving Message Types #358

Open
wants to merge 22 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
df394d9
WIP
wjwwood Feb 15, 2022
c355ca6
still working
wjwwood Mar 9, 2022
05ac47e
added more about the enforcing of the type versions
wjwwood Apr 13, 2022
6dcb876
Increase readability, fix typos, and reduce ambiguities
methylDragon May 13, 2022
42cf0de
Add clarification for field semantics
methylDragon May 16, 2022
d3449f0
Implement more fixups
methylDragon Jun 1, 2022
98081d9
Methyl dragon/type description distribution (#2)
wjwwood Jun 10, 2022
d7628e7
rename from REP-2010 to REP-2011
wjwwood Jun 10, 2022
22f6cb7
Add runtime type introspection section (#3)
methylDragon Jul 8, 2022
8dccc0f
Add nested type description section (#4)
methylDragon Jul 23, 2022
b6a7086
Add alternatives and on_inconsistent_topic section (#5)
youliangtan Jul 23, 2022
a030033
fix rst errors, revise a few sections
wjwwood Jul 26, 2022
04306e4
touch ups, rename to Run-Time Interface Reflection, reorganize
wjwwood Jul 27, 2022
d5e27d5
diagram typo
wjwwood Jul 27, 2022
fbc8810
fixup
wjwwood Aug 1, 2022
01edda8
change title to a better wording
wjwwood Aug 2, 2022
9ea2a20
fixups, reordering, tools section
wjwwood Aug 22, 2022
24492e5
add feature progress section
wjwwood Sep 6, 2022
bd5cdbc
cleanup feature progress list
wjwwood Sep 9, 2022
ae78cf9
Update Type Version Hash to use SHA-256, and details updates based on…
emersonknapp Jan 31, 2023
ba2f96c
Update RIHS notes based on latest discussion (#8)
emersonknapp Mar 29, 2023
a4e1ef5
Remove Type Description/Hash content (moving to REP-2016) (#9)
emersonknapp Sep 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update RIHS notes based on latest discussion (#8)
Signed-off-by: Emerson Knapp <[email protected]>
emersonknapp authored Mar 29, 2023
commit ba2f96c5163b77385956d51065607201ae3f71e7
16 changes: 10 additions & 6 deletions rep-2011.rst
Original file line number Diff line number Diff line change
@@ -274,9 +274,13 @@ Finally, the resulting filled data structure must be represented in a platform-i
Different languages, architectures, or compilers will produce different in-memory representations, and the hash must be consistently calculable in different contexts.

The resulting data structure is hashed using SHA-256, resulting in a 256-bit (32-byte) hash value which is also generally known as a "message digest".
This hash is combined with a type version hash standard version, which we will call the "ROS IDL Hashing Standard" or "RIHS", the first version of which will be ``RIHS1``.
They are combined using an ``_`` symbol, resulting in a complete type version hash like ``RIHS1_<256-bit SHA-256 of data structure>``.
This allows the tooling to know if a hash mismatch is due to a change in this standard (how hash is computed) or due to a difference in the interface types themselves.
This hash is paired with a type version hash standard version, which we will call the "ROS IDL Hashing Standard" or "RIHS", the first version of which will be ``RIHS01``.
RIHS Version 00 is reserved for "Invalid" / "unset", and the RIHS version is limited by this specification to a maximum value of 255.
RIHS hash values must have a well-defined UTF-8 string representation for human readability and for passing over string-only communication channels.
The prefix of a well-formed RIHS string will always be ``RIHSXX_``, where ``X`` is one hexadecimal digit, followed by the version-dependent string representation of the hash value.
For ``RIHS01``, the hash value is 64 hexadecimal digits representing the 256-bit message digest, leading to a known ``RIHS01`` string length of 71.

This versioning allows the tooling to know if a hash mismatch is due to a change in this standard (how hash is computed) or due to a difference in the interface types themselves.
In the case of a change in standard, it will be unknown whether the interface types are equal or not.

For now, the list of field names and their types are the only contributing factors, but in the future that could change, depending on which "annotations" are supported in ``.idl`` files.
@@ -348,7 +352,7 @@ For example, a type name using this approach may look like this:

.. code::

sensor_msgs/msg/Image__RIHS1_XXXXXXXXXXXXXXXXXXXX
sensor_msgs/msg/Image__RIHS01_XXXXXXXXXXXXXXXXXXXX

This has the benefit of "just working" for most middlewares which at least match based on the name of the type, and it is simple, requiring no further custom hooks into the middleware's discovery or matchmaking process.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any consideration to be made on the max. lengths of the type names? Any restrictions to consider? Given the concatenated type version hash there might be little margin for using namespaces (which larger graphs tend to use).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point, I suppose we'd have to consider the DDSI-RTPS spec that limits topic lengths to 256 characters...

Though, looking at what such a character length even looks like:

sensor_msgs/msg/Image__RIHS1_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

I think there's plenty of space for multiple nested namespaces? I'd imagine the type version hash wouldn't need more than 32 characters to have a relatively low chance of collision, but that's just a guess off the top of my head.

In which case a topic name would have 224 characters, which is still a lot.

sensor_msgs/msg/Image__RIHS1_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, since we're speaking about type names here (not topic names), the convention is to use <package name>/<type, msg/srv/action>/<name>, and so deeply nested namespaces isn't really as big of an issue as it is with things like topic names and node names.

In table 9.12 of the DDSI-RTPS 2.2 spec (https://www.omg.org/spec/DDSI-RTPS/2.2/PDF) it says the type name is limited to 256 characters as well. So the limit is the same, but again, I don't think we're in danger here of running into this limitation in practical situations. At least not very often.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One exception to this is the feature that allows you to configure topic endpoint's QoS using parameters: http://design.ros2.org/articles/qos_configurability.html

In this case the topic name is part of the parameter name, but unlike topic/service/node names, we don't have a limit for the length of parameter names at the moment.


@@ -776,8 +780,8 @@ Registering a transfer function in an ``ament_cmake`` project might look like th
create_transfer_function(
src/my_transfer_function.cpp
FUNCTION_NAME my_transfer_function
FROM sensor_msgs/msg/Image RIHS1_XXXXXXXXXXXXXXXXX123
TO sensor_msgs/msg/Image RIHS1_XXXXXXXXXXXXXXXXXabc
FROM sensor_msgs/msg/Image RIHS01_XXXXXXXXXXXXXXXXX123
TO sensor_msgs/msg/Image RIHS01_XXXXXXXXXXXXXXXXXabc
)

This CMake function would create the library target, link the necessary libraries, and install any files in the ``ament_index`` needed to make the transfer function discoverable by the tools.