Skip to content

Commit

Permalink
[doc][log] Add new roadmap (again) #69
Browse files Browse the repository at this point in the history
  • Loading branch information
at15 committed Jan 17, 2020
1 parent 9415607 commit 45fc69a
Show file tree
Hide file tree
Showing 2 changed files with 46 additions and 1 deletion.
45 changes: 45 additions & 0 deletions doc/log/2020/2020-01/2020-01-17-refactor-again.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# 2020-01-17 Refactor again

It's almost (exactly) 1 year after [previous effort for refactoring](../../2019/2019-01/2019-01-25-0.0.3-init-refactor.md) (well rewrite).
I have to say although I didn't write any code, I did write a detailed plan in [#69](https://github.com/xephonhq/xephon-k/issues/69).

## Design

I'd start with the simplest data model and single node

Data model

- tagged time series like Prometheus and InfluxDB

Storage engine

- assuming tag index can fit into memory (obviously it may not be the case)
- use simple columnar store, i.e. just blocks.
- support WAL but can be disabled (mainly used to see the effect of having both WAL and data on single disk)
- support compaction on single node

Compression

- relies on libtsdb-go

Protocol

Each protocol should have a prepared statement style variant (because is might be much faster? though gzip entire payload could beat it?)

- http json (kairosdb like)
- tcp, text and binary
- grpc
- prometheus

Query

- filter by tag and returns as it is, i.e. no aggregation, no group by. Mainly used for verify correctness

## Dependencies

- compression in libtsdb
- need to move existing simple compression code to libtsdb, and implement gorilla tsz
- error and logging from gommon
- tracing?
- I want to have a simple in process tracing, it may not be that feature rich [tokio-rs/tracing](https://github.com/tokio-rs/tracing) might be a good reference.
- end to end benchmark from xephon-b
2 changes: 1 addition & 1 deletion playground/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Playground

For testing language semantics and small benchmarks.
For testing language semantics and do microbenchmark.
Prototype and library examples are also put here.

## Language
Expand Down

0 comments on commit 45fc69a

Please sign in to comment.