flash-store

flash-store

FlashStore is Key-Value persistent storage with easy to use ES6 Map-like API(both Async and Sync support), powered by LevelDB and TypeScript.

Requirements

1. Node.js v10 or above

Examples

Try the following command

npm install
npm run demo

The basic function as follows:

import { FlashStore } from 'flash-store'

const flashStore = new FlashStore('flashstore.workdir')

await flashStore.set(1, 'a')
console.log(`key: 1, value: ${await flashStore.get(1)}`)
// Output: 'a'

await flashStore.del(1)
console.log(`key: 1, value: ${await flashStore.get(1)}`)
// Output: undefined

Supported Backend

Backend	Flash Store	Install NPM Command
LevelDB	v1.0	npm install flash-store@1
SQLite	v0.20	npm install flash-store@0.20
Medea	v0.18	npm install flash-store@0.18 (deprecated)
SnapDB	v0.16	npm install flash-store@0.16 (deprecated)
RocksDB	v0.14	npm install flash-store@0.14
LevelDB	v0.12	npm install flash-store@0.12

SnapDB & Medea were all deprecated because of lots of unknown bugs.

API Reference

FlashStore

FlashStore implements the Standard ES6 Map API with Async modification, powered by async-map-like

/**
 * ES6 Map API with Async
 */
export interface AsyncMap<K = any, V = any> {
  [Symbol.asyncIterator]() : AsyncIterableIterator<[K, V]>
  size                     : Promise<number>

  clear   ()                 : Promise<void>
  delete  (key: K)           : Promise<void>
  entries()                  : AsyncIterableIterator<[K, V]>
  get     (key: K)           : Promise<V | undefined>
  has     (key: K)           : Promise<boolean>
  keys    ()                 : AsyncIterableIterator<K>
  set     (key: K, value: V) : Promise<void>
  values  ()                 : AsyncIterableIterator<V>
}

class FlashStore<K, V> implements AsyncMap<K, V> {}

FlashStoreSync

FlashStoreSync implements the Standard ES6 Map API:

class FlashStoreSync<K, V> implements Map<K, V> {}

1. You get a sync API at the cost of all your data have to be kept in memory.
2. The data will be async writing back to disk for persistent storage in background.
3. The performance of FlashStoreSync can be expected high because it's all in memory.

Document

See auto generated docs

• ECMAScript 6: Maps - 2ality – JavaScript and more

See Also

1. Node.js databases: an embedded database using LevelDB
2. How to Cook a Graph Database in a Night - LevelGraph
3. Graph database JS style for Node.js and the Browser. Built upon LevelUp and LevelDB.
4. 浅析 BigTable 和 LevelDB 的实现

Known Issues

1. The gte and lte in options do not work property. (#4)

Version History

master

v1.0 (Aug 16, 2021) LevelDB v7.0

LevelDB v7.0.0 support.

v0.20 Apr 2020 SQLite as Backend

1. We hardcoded the key type to be string only in this version.
2. We decide to try better-sqlite3 as it claim is very fast.
3. The other alternates (would love to try in the future if necessary):
   1. TypeScript: sqlite
   2. WebAssembly: sql.js

v0.18 Feb 2019 - Medea as Backend

DEPRECATED: Due to #79 #74 and also it is very unstable in production as my experiences. (e.g. memory leak & block event loop)

1. Switch from SnapDB to MedeaDown

Medea is a persistent key-value storage library that runs everywhere Node runs.

"It is a pure JS implementation of leveldown and it's almost as fast." — @Raynos link

"The LevelDOWN-compatible wrapper for Medea really opens up the possibility to reuse the modules that have already been created around the LevelUP ecosystem." — @kevinswiber link

Known Issues: FlashStore 会写满磁盘的问题 #155

async function compact (store: FlashStore): Promise<void> {
  await store.size
  const db = (store as any).levelDb.db.db.db
  await new Promise((resolve, reject) => {
    db.compact((err: any) => {
      if (err) {
        return reject(err)
      }
      resolve()
    })
  })
}

v0.16 May 2019 - SnapDB as Backend

1. Switch from RocksDB to SnapDB #45
2. #50 has been fixed. WARN: Do not use this version because it has known issues

v0.14 May 2019 - RocksDB as Backend

1. Switch from LevelDB to RocksDB #34

v0.12 Jan 2019 - LevelDB as Backend

1. Use LevelDB as backend to skip the compiling when install.
2. Using leveldb official typings from @types/

v0.7 Aug 2018 - Nosql-LevelDB as Backend

1. Use nosql-leveldb as backend to prevent segfault.

v0.6 Jul 2018

1. Upgrade to TypeScript 3.0

v0.4 Jun 2018

1. Refactor API to implement ES6 Map interface

1. Update the API to ES6 Map-like, the difference is that FlashStore is all async.

2. Add FlashStoreSync as a in-memory Write-back Cache for Flash-Store

Add a new class FlashStoreSync which is a in-memory full loaded Write-back Cache for Flash-Store:

1. Writes directly to cache, lazy writes to store.
2. Reads from cache, never read-miss because cache have the full data of the store which will never expire.
3. API of FlashStoreSync is the same as the ES6 Map

v0.2 Sep 2017

Init version, API is LevelDB-like.

FAQ

Q: What's the difference between the flash-store and memory-card

Short answer:

1. flash-store is for save data to local filesystem.
2. memory-card is for save data to a distributed network storage, it can be serialized/deserialized safely by design.

Long answer:

flash-store and memory-card are all built by @huan, and they are all follow the ES6 Map API.

flash-store is using a no-sql local file database to maximum the performance, it can be used as a local database, or a local cache for whatever you want to cache from other API.

memory-card is using a local file to store data in JSON format by default, however, it supports more distributed methods. Learn more from it's repository at here

Author

Huan LI (李卓桓) [e-mail]

Copyright & License

• Code & Docs © 2017-now Huan (李卓桓) <[e-mail]>
• Code released under the Apache-2.0 License
• Docs released under Creative Commons