From 45fc69acaf94bd12839c5ff008e1c3bafc498688 Mon Sep 17 00:00:00 2001
From: at15 <at15@outlook.com>
Date: Fri, 17 Jan 2020 13:27:54 -0800
Subject: [PATCH] [doc][log] Add new roadmap (again) #69

---
 .../2020/2020-01/2020-01-17-refactor-again.md | 45 +++++++++++++++++++
 playground/README.md                          |  2 +-
 2 files changed, 46 insertions(+), 1 deletion(-)
 create mode 100644 doc/log/2020/2020-01/2020-01-17-refactor-again.md

diff --git a/doc/log/2020/2020-01/2020-01-17-refactor-again.md b/doc/log/2020/2020-01/2020-01-17-refactor-again.md
new file mode 100644
index 0000000..3535b40
--- /dev/null
+++ b/doc/log/2020/2020-01/2020-01-17-refactor-again.md
@@ -0,0 +1,45 @@
+# 2020-01-17 Refactor again
+
+It's almost (exactly) 1 year after [previous effort for refactoring](../../2019/2019-01/2019-01-25-0.0.3-init-refactor.md) (well rewrite).
+I have to say although I didn't write any code, I did write a detailed plan in [#69](https://github.com/xephonhq/xephon-k/issues/69).
+
+## Design
+
+I'd start with the simplest data model and single node
+
+Data model
+
+- tagged time series like Prometheus and InfluxDB
+
+Storage engine
+
+- assuming tag index can fit into memory (obviously it may not be the case)
+- use simple columnar store, i.e. just blocks.
+- support WAL but can be disabled (mainly used to see the effect of having both WAL and data on single disk)
+- support compaction on single node
+
+Compression
+
+- relies on libtsdb-go
+
+Protocol
+
+Each protocol should have a prepared statement style variant (because is might be much faster? though gzip entire payload could beat it?)
+
+- http json (kairosdb like)
+- tcp, text and binary
+- grpc
+- prometheus
+
+Query
+
+- filter by tag and returns as it is, i.e. no aggregation, no group by. Mainly used for verify correctness
+
+## Dependencies
+
+- compression in libtsdb
+  - need to move existing simple compression code to libtsdb, and implement gorilla tsz
+- error and logging from gommon
+- tracing?
+  - I want to have a simple in process tracing, it may not be that feature rich [tokio-rs/tracing](https://github.com/tokio-rs/tracing) might be a good reference.
+- end to end benchmark from xephon-b
\ No newline at end of file
diff --git a/playground/README.md b/playground/README.md
index 9661fcf..59c3074 100644
--- a/playground/README.md
+++ b/playground/README.md
@@ -1,6 +1,6 @@
 # Playground
 
-For testing language semantics and small benchmarks.
+For testing language semantics and do microbenchmark.
 Prototype and library examples are also put here.  
 
 ## Language