github.com/tsandl/go-wiskey-update

go-wiskey

Golang implementation of Wiskey paper

Description

Dead simple lsm implementation which stores values in the vlog which decreases write amplification of lsm tree during merging

Things that are implemented

• SSTable
  • Create sstable
  • Read from sstable
• Memtable(in memory redblack tree that stores the data and flushes it once memory is full)
  • Put
  • Delete
  • Get
• Lsm tree
  • Put
  • Get
  • Delete
• Http interface
  • Http Get
  • Http Put
  • Http Delete
• Crash recovery
  • Store the last head position in the separate file
  • Store al values from head to tail into the memtable during recovery
• Merge sstable files
• Cli interface
  • specify sstable path
  • specify vlog path
  • specify checkpoint path
  • specify memtable size
• Reclaim space
  • Merge sstables
  • Garbage collect vlog

Install

In order to install the binary run go get github.com/strogiyotec/go-wiskey , it will be installed in $HOME/go/bin/wiskey

Usage

In order to start the app run wiskey -s ../go-wiskey/sstable -v vlog -c checkpoint -m 20 where :

• -s - directory with sstables
• -v - path to vlog file(vlog doesn't have to exist)
• -c - path to checkpoint (checkpoint doesn't have to exist)
• -m - memtable size in bytes(the size of in memory red black tree that keeps keys , when full will flush this tree to sstable)

It will start an http server

Http server

In order to GET/UPDATE/DELETE you can use http endpoints

• Save key value
  • curl -X POST -H "Content-Type: application/json" -d '{"value":"Developer"}' http://localhost:8080/anita it will save value Developer with a key anita
• Get by key - curl -i localhost:8080/fetch/anita
• Delete by key - curl -i localhost:8080/fetch/anita

How it works

Here is the general image on how the storage works