Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[reboot] Add dev log and roadmap #69

Open
19 tasks
at15 opened this issue Jan 15, 2019 · 0 comments
Open
19 tasks

[reboot] Add dev log and roadmap #69

at15 opened this issue Jan 15, 2019 · 0 comments
Assignees

Comments

@at15
Copy link
Member

at15 commented Jan 15, 2019

It's been a while since I worked on xephon ... a lot has changed except the fact I never finish the projects I started ... Anyway, even go.ice is start being maintained, so xephon-k should get revived as well. Also there are more tsdb written in go now.

TODO

  • init log to recap what has been done in xephon-k (good old grad school memories)
  • start playground to test missed things
    • consistent hashing
    • serialize inverted index (was building from scratch every time)
    • compress inverted index
    • more general data model, druid, piont
    • compaction
    • different transport, grpc was never really supported
    • tcp based line protocol
    • different compression (existing, RLE, delta RLE are too simple ...)
    • consider use tree for aggregation
  • clean up old codebase, I think most of them are no longer useful in the new code base beyond reference ...
  • a spec for implementation in other language especially java, c++

Dependencies

  • reboot awesome-tsdb, it's been dead for even longer .... my ‘I will finish it next week' is still there ...
  • go.ice, gommon need to change accordingly, with dep it should be much easier using replace directive

A MVP would be

  • store in memory, inverted index + series
  • WAL (I still don't think WAL is that useful for large volume similar time series data), but adding WAL is much harder than remove it ... so this time, do things in right order
  • compaction
  • use consistent hashing to shard series by meta (tags)
@at15 at15 self-assigned this Jan 15, 2019
at15 added a commit that referenced this issue Jan 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant