rust: add async reading functionality

[james-rms]

Changelog

• rust: switches the LZ4 compression dependency from lz4_flex to lz4-rs. This moves us from using a pure-rust lz4 implementation to C bindings. I believe this is worthwhile because lz4_flex does not support LZ4 "high compression mode". The practical reason for doing so in this PR is that lz4_flex does not expose interfaces that make it easy to build an AsyncRead adapter for it, but lz4-rs does.
• rust: Adds structs to read MCAP data asynchronously in a linear stream.

Docs

• Check generated rust docs for review.

Description

Adds an async RecordReaderimplementation, for reading MCAP data asynchronously. This is an optional feature, named tokio. I chose this feature flag name and this module name because this functionality is tied heavily into the Tokio ecosystem. If at some point we rebuild this to be async-executor-agnostic, we can add that functionality under a new module and feature flag name.

Before	After

[james-rms]

@mrkline do you have strong opinions on lz4-flex vs lz4?

[eloff]

I think this needs one change to a test, and the rest is optional

[eloff]

The problem with this is it fills the buffer in one shot. So the read loop doesn't ever run more than once. It would be a better test if it read e.g. a byte at a time into the buffer.

[james-rms]

I don't get it - the reader doesn't fill the buffer in one shot, it only yields up to max_read_len at a time. Is there something i'm missing?

[eloff]

So it reads up to max_read_len bytes, which in the tests would usually fit in the buffer perfectly, so it exits the read loop without trying to read more (reads all the data on one pass) so if there was an issue with the subsequent iterations of the read loop, as would be exercised in the real world, then you wouldn't see it with this test.

It would be a better test if this required at least two read calls to get all the data, then the caller would have to loop twice at least and exercise that code.

Speaking from long experience of having had a bug exactly like this that wasn't caught by the tests.

[eloff]

I'm going to approve the PR so this doesn't hold up your work, it's up to you if you want to improve on the test or not