Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
@TaskerJang
TaskerJang authored Feb 23, 2025
1 parent 65214a0 commit 9e00286
Showing 1 changed file with 122 additions and 184 deletions.
306 changes: 122 additions & 184 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,219 +1,157 @@
# raftify
# 🚀 Raftify: Rust 기반 분산 합의 시스템

⚠️ WARNING: This library is in a very experimental stage. The API could be broken.
## 🎯 소개
2024 오픈소스 컨트리뷰션 아카데미(OSSCA) 프로젝트로, Raft 합의 알고리즘의 Rust 구현체에 RocksDB 스토리지를 통합했습니다.

raftify is a *high-level* implementation of [Raft](https://raft.github.io/), developed with the goal of making it easy and straightforward to integrate the Raft algorithm.
### ⏰ 프로젝트 기간
- **전체**: 2024.7.13 - 2024.11.2 (13주)
- **Challenges**: 2024.7.13 - 2024.8.9
- **Masters**: 2024.8.10 - 2024.11.2

It uses [tikv/raft-rs](https://github.com/tikv/raft-rs) and gRPC for the network layer and [heed](https://github.com/meilisearch/heed) (LMDB wrapper) for the storage layer.
## 💻 기술 스택
- **언어**: Rust
- **스토리지**: RocksDB
- **통신**: gRPC
- **웹 프레임워크**: Actix-web
- **직렬화**: Protobuf, Serde

## Quick guide
## 🔧 구현 내용

I strongly recommend to read the [basic memstore example code](https://github.com/lablup/raftify/blob/main/examples/memstore/static-members/src/main.rs) to get how to use this library for starters, but here's a quick guide.

### Define your own log entry

Define the data to be stored in `LogEntry` and how to *serialize* and *deserialize* it.
### 1. Raft 알고리즘 구현
#### 핵심 컴포넌트
- **리더 선출**: 랜덤화된 타이머 기반 선출 시스템
- **로그 복제**: 리더 주도 로그 동기화
- **상태 관리**: Term 기반 상태 추적

#### 주요 특징
```rust
#[derive(Clone, Debug, Serialize, Deserialize)]
pub enum LogEntry {
Insert { key: u64, value: String },
}

impl AbstractLogEntry for LogEntry {
fn encode(&self) -> Result<Vec<u8>> {
serialize(self).map_err(|e| e.into())
}

fn decode(bytes: &[u8]) -> Result<LogEntry> {
let log_entry: LogEntry = deserialize(bytes)?;
Ok(log_entry)
}
pub trait StableStorage: Storage {
const STORAGE_TYPE: StorageType;
fn append(&mut self, entries: &[Entry]) -> Result<()>;
fn hard_state(&self) -> Result<HardState>;
// ... 기타 메서드
}
```

### Define your application Raft FSM

Essentially, the following three methods need to be implemented for the `Store`.

- `apply`: applies a committed entry to the store.
- `snapshot`: returns snapshot data for the store.
- `restore`: applies the snapshot passed as argument.

And also similarly to `LogEntry`, you need to implement `encode` and `decode`.
### 2. RocksDB 스토리지 구현

#### 핵심 기능
```rust
#[derive(Clone, Debug)]
pub struct HashStore(pub Arc<RwLock<HashMap<u64, String>>>);

impl HashStore {
pub fn new() -> Self {
Self(Arc::new(RwLock::new(HashMap::new())))
}

pub fn get(&self, id: u64) -> Option<String> {
self.0.read().unwrap().get(&id).cloned()
}
pub struct RocksDBStorage {
db: DB,
logger: Arc<dyn Logger>,
}

#[async_trait]
impl AbstractStateMachine for HashStore {
async fn apply(&mut self, data: Vec<u8>) -> Result<Vec<u8>> {
let log_entry: LogEntry = LogEntry::decode(&data)?;
match log_entry {
LogEntry::Insert { ref key, ref value } => {
let mut db = self.0.write().unwrap();
log::info!("Inserted: ({}, {})", key, value);
db.insert(*key, value.clone());
}
};
Ok(data)
}

async fn snapshot(&self) -> Result<Vec<u8>> {
Ok(serialize(&self.0.read().unwrap().clone())?)
}

async fn restore(&mut self, snapshot: Vec<u8>) -> Result<()> {
let new: HashMap<u64, String> = deserialize(&snapshot[..]).unwrap();
let mut db = self.0.write().unwrap();
let _ = std::mem::replace(&mut *db, new);
Ok(())
}

fn encode(&self) -> Result<Vec<u8>> {
serialize(&self.0.read().unwrap().clone()).map_err(|e| e.into())
}

fn decode(bytes: &[u8]) -> Result<Self> {
let db: HashMap<u64, String> = deserialize(bytes)?;
Ok(Self(Arc::new(RwLock::new(db))))
impl RocksDBStorage {
pub fn create(path: &str, logger: Arc<dyn Logger>) -> Result<Self> {
let mut opts = Options::default();
opts.create_if_missing(true);
let db = DB::open(&opts, path)?;
Ok(RocksDBStorage { db, logger })
}
}
```

### Bootstrap a raft cluster

First bootstrap the cluster that contains the leader node.
#### 주요 컴포넌트
1. **Write Ahead Logging (WAL)**
- 데이터 영속성 보장
- 장애 복구 메커니즘

```rust
let raft_addr = "127.0.0.1:60061".to_owned();
let node_id = 1;

let log_storage = HeedStorage::create(&storage_pth, &raft_config.clone(), logger.clone())
.expect("Failed to create heed storage");

let raft = Raft::bootstrap(
node_id,
raft_addr,
log_storage,
store.clone(),
raft_config,
logger.clone(),
)?;

tokio::spawn(raft.clone().run());

// ...
tokio::try_join!(raft_handle)?;
```
2. **MemTable**
- 인메모리 데이터 구조
- 빠른 읽기/쓰기 지원

### Join follower nodes to the cluster
3. **SST (Sorted String Table) Files**
- 계층화된 데이터 저장
- 효율적인 압축

Then join the follower nodes.
4. **Compaction**
- 자동 로그 압축
- 스토리지 최적화

If peer specifies the configuration of the initial members, the cluster will operate after all member nodes are bootstrapped.
5. **Bloom Filter**
- 효율적인 키 검색
- 불필요한 디스크 접근 방지

### 3. HTTP API 개선
```rust
let raft_addr = "127.0.0.1:60062".to_owned();
let peer_addr = "127.0.0.1:60061".to_owned();
let join_ticket = Raft::request_id(raft_addr, peer_addr).await;

let log_storage = HeedStorage::create(&storage_pth, &raft_config.clone(), logger.clone())
.expect("Failed to create heed storage");

let raft = Raft::bootstrap(
join_ticket.reserved_id,
raft_addr,
log_storage,
store.clone(),
raft_config,
logger.clone(),
)?;

let raft_handle = tokio::spawn(raft.clone().run());
raft.join_cluster(vec![join_ticket]).await?;

// ...
tokio::try_join!(raft_handle)?;
```

### Manipulate FSM by RaftServiceClient

If you want to operate the FSM remotely, you can use [RaftServiceClient](https://docs.rs/raftify/latest/raftify/raft_service/raft_service_client/struct.RaftServiceClient.html).
#[post("/leave")] // 상태 변경 작업은 POST로 수정
async fn leave(data: web::Data<(HashStore, Raft)>) -> impl Responder {
let raft = data.clone();
raft.1.leave().await.unwrap();
"OK".to_string()
}

```rust
let mut leader_client = create_client(&"127.0.0.1:60061").await.unwrap();

leader_client
.propose(raft_service::ProposeArgs {
msg: LogEntry::Insert {
key: 1,
value: "test".to_string(),
}
.encode()
.unwrap(),
})
.await
.unwrap();
#[get("/status")] // 상태 조회는 GET 유지
async fn status(data: web::Data<(HashStore, Raft)>) -> impl Responder {
// ... 상태 조회 로직
}
```

### Manipulate FSM by RaftNode
## 📦 설치 및 실행

If you want to operate FSM locally, use the [RaftNode](https://docs.rs/raftify/latest/raftify/struct.RaftNode.html) type of the [Raft](https://docs.rs/raftify/latest/raftify/struct.Raft.html) object.

```rust
raft.propose(LogEntry::Insert {
key: 123,
value: "test".to_string(),
}.encode().unwrap()).await;
### 의존성 설정
```toml
[dependencies]
raftify = { version = "0.1.78", features = ["rocksdb_storage"] }
rocksdb = "0.19.0"
```

## Debugging
### 스토리지 설정
```toml
[features]
default = ["heed_storage"]
rocksdb_storage = ["rocksdb"]
inmemory_storage = []
heed_storage = ["heed", "heed-traits"]
```

You can use a collection of CLI commands that let you inspect the data persisted in stable storage and the status of Raft Servers.
## 🏗 아키텍처

### 시스템 구조
```
❯ raftify-cli debug persisted ./logs/node-1
---- Persisted entries ----
Key: 1, "Entry { context: [], data: [], entry_type: EntryNormal, index: 1, sync_log: false, term: 1 }"
Key: 2, "Entry { context: [], data: ConfChange { change_type: AddNode, node_id: 2, context: [127.0.0.1:60062], id: 0 }, entry_type: EntryConfChange, index: 2, sync_log: false, term: 1 }"
Key: 3, "Entry { context: [], data: ConfChange { change_type: AddNode, node_id: 3, context: [127.0.0.1:60063], id: 0 }, entry_type: EntryConfChange, index: 3, sync_log: false, term: 1 }"
---- Metadata ----
HardState { term: 1, vote: 1, commit: 3 }
ConfState { voters: [1, 2, 3], learners: [], voters_outgoing: [], learners_next: [], auto_leave: false }
Snapshot { data: HashStore(RwLock { data: {}, poisoned: false, .. }), metadata: Some(SnapshotMetadata { conf_state: Some(ConfState { voters: [1, 2, 3], learners: [], voters_outgoing: [], learners_next: [], auto_leave: false }), index: 1, term: 1 }) }
Last index: 3
src/
├── storage/
│ ├── rocksdb_storage/ # RocksDB 구현
│ ├── heed_storage/ # Heed 구현
│ └── inmemory_storage/ # 인메모리 구현
├── raft/ # Raft 알고리즘 코어
└── api/ # HTTP API 레이어
```

## Bootstrapping from WAL

If there are previous logs remaining in the log directory, the raft node will automatically apply them after the node is bootstrapped.
If you intend to bootstrap the cluster from the scratch, please remove the previous log directory.
To ignore the previous logs and bootstrap the cluster from a snapshot, use the `Config.bootstrap_from_snapshot` option.

## Support for other languages

raftify provides bindings for the following languages.

- [Python](https://github.com/lablup/raftify/tree/main/binding/python)

## References

raftify was inspired by a wide variety of previous Raft implementations.

Great thanks to all the relevant developers.

- [tikv/raft-rs](https://github.com/tikv/raft-rs) - Raft distributed consensus algorithm implemented using in this lib under the hood.
- [ritelabs/riteraft](https://github.com/ritelabs/riteraft) - A raft framework, for regular people. raftify was forked from this lib.
## 📚 기술 문서

### RocksDB 관련
- [Write Ahead Logging](https://velog.io/@tasker_dev/Write-Ahead-Logging)
- [MemTable 구조](https://velog.io/@tasker_dev/MemTable)
- [SST Files](https://velog.io/@tasker_dev/SST-Files)
- [Compaction](https://velog.io/@tasker_dev/Compaction)
- [Bloom Filter](https://velog.io/@tasker_dev/Bloom-Filter)
- [Transaction Log](https://velog.io/@tasker_dev/11.-Transaction-Log)

### Raft 알고리즘
- [Raft 배경 지식](https://velog.io/@tasker_dev/1.-러스트에-들어가기-앞서-배경지식)
- [합의 알고리즘 이해](https://velog.io/@tasker_dev/In-Search-of-an-Understandable-Consensus-Algorithm)
- [코드 분석](https://velog.io/@tasker_dev/Raft-코드-뜯어보기)

## 🎉 프로젝트 성과

### Pull Requests
1. [#124 RocksDB 스토리지 구현](https://github.com/lablup/raftify/pull/124)
- RocksDB 백엔드 통합
- 성능 최적화
- 테스트 케이스 구현

2. [#147 HTTP API 개선](https://github.com/lablup/raftify/pull/147)
- RESTful 원칙 준수
- API 일관성 향상

## 🔜 향후 계획
- Java 기반 프로젝트 참여
- Raftify 지속적 개선
- 성능 최적화 작업

## 📝 참고 자료
- [RocksDB 공식 문서](https://rocksdb.org)
- [Raft 논문](https://raft.github.io)
- [프로젝트 블로그](https://velog.io/@tasker_dev)

0 comments on commit 9e00286

Please sign in to comment.