LSM based key-value store in rust, design for cloud.
This project is in Alpha phase. API and data layout will change rapidly.
-
Transactional key-value store: Provide transactional write, and snapshot read.
-
Pluggable WAL: using local fs, raft log, or Kafka as a WAL provider.
-
Minimize ser/de cost: Access to serialized data in SSTable without deserializing.
-
Design for large datasets:
Reduce unnecessary IO with an efficient data index. Reduce in-memory data size by combining sparse-index and bloom filter.
-
Flexible compaction policy:
Reduces unnecessary compaction by comparing SSTable cardinality signature. Reduces IO consumption with partially SSTable merge down. Reclaims space quickly with cross-level SSTable merge.
-
Design for the cloud: Stores less accessed SSTable on S3.
-
Internal sharding A flattened structure reduces write/read amplification. A span with a heavy load will be pushed down more frequently than other spans.