Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic streams implementation #110

Closed
wants to merge 4 commits into from

Conversation

skateinmars
Copy link
Contributor

@skateinmars skateinmars commented Oct 26, 2019

This PR adds a few commands related to streams:

  • XADD
  • XLEN
  • XRANGE
  • XREVRANGE

Overall the code is heavily inspired from the sorted sets implementation.
The data structure used to store streams on the DB is a slice of streamEntry which are composed of a pair of uint64 for the ID, and a [][2]string for the values to guarantee the insertion order.

"Direct" functions have been added, namely XAdd and Stream. Note that XAdd behaves differently from the XADD command since it allows for adding items with an ID inferior to the latest existing ID. I figured this would be useful for writing tests?

XADD uses time.Now() when autogenerating an ID, which replicates Redis's behavior.

I did not implement the MAXLEN option of XADD, so a custom error is returned for now. Implementing it should be straightforward along with XDEL and XTRIM.

XLEN, XRANGE and XREVRANGE should be identical feature-wise to Redis.
The range commands are more heavily tested via integration tests since redigo does not provide native support for streams (see gomodule/redigo#375).

Fixes #57 !

@skateinmars skateinmars changed the title Baisc streams implementation Basic streams implementation Oct 26, 2019
@alicebob
Copy link
Owner

alicebob commented Oct 28, 2019 via email

This way the IDs themselves are predictable and testable.
@alicebob
Copy link
Owner

alicebob commented Nov 6, 2019

Hi @skateinmars !

I merged if with some changes:

  • the direct XAdd() must now obey the same rules for keys as XADD (simpler implementation)
  • I changed to stream element returned by m.Stream(key) to be a slice of StreamEntrys, which is a simple struct:
type StreamEntry struct {
    ID     string
    Values []string
}

That seemed simpler than a slice of maps of a slice of tuples. Also the Values turned into a simple slice, since that's what redis also returns and is simpler.

  • XADD MAXLEN is also there now :)

It's now in master, I plan to make it a proper release in a few days.

@alicebob alicebob closed this Nov 6, 2019
@skateinmars
Copy link
Contributor Author

skateinmars commented Nov 6, 2019

the direct XAdd() must now obey the same rules for keys as XADD (simpler implementation)

This was a bit of an obscure choice so this makes more sense

Values turned into a simple slice, since that's what redis also returns and is simpler.

Right, this is indeed closer to what clients get from redis

Thanks for the merge! 🤘

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[feature] support streams
2 participants