level-ws

What is level-ws?

The level-ws npm package is a WriteStream implementation for LevelDB, which allows for efficient batch writing of data to a LevelDB database. It is particularly useful for handling large amounts of data that need to be written to the database in a streaming fashion.

What are level-ws's main functionalities?

Batch Writing

This feature allows you to write multiple key-value pairs to a LevelDB database in a batch. The code sample demonstrates how to create a WriteStream and write multiple entries to the database.

const level = require('level');
const levelws = require('level-ws');

const db = level('./mydb');
const ws = levelws(db);

ws.write({ key: 'name', value: 'Alice' });
ws.write({ key: 'age', value: '30' });
ws.end();

Stream Handling

This feature allows you to handle streams of data and pipe them directly into the LevelDB database. The code sample demonstrates how to create a readable stream and pipe it into the WriteStream.

const level = require('level');
const levelws = require('level-ws');
const { Readable } = require('stream');

const db = level('./mydb');
const ws = levelws(db);

const data = [
  { key: 'name', value: 'Alice' },
  { key: 'age', value: '30' }
];

const readable = Readable.from(data);
readable.pipe(ws);

Other packages similar to level-ws

levelup

LevelUP is a Node.js module that provides a higher-level API for interacting with LevelDB. It offers a more user-friendly interface compared to the lower-level LevelDB bindings. While level-ws focuses on WriteStream functionality, LevelUP provides a broader range of database operations including read, write, delete, and batch operations.

leveldown

LevelDOWN is the backend store for LevelDB in Node.js. It provides the low-level bindings to the LevelDB library. While level-ws is specifically for streaming writes, LevelDOWN offers the foundational operations for interacting with LevelDB, such as put, get, and del.

level

The level package is a convenience package that bundles LevelUP and LevelDOWN, providing a simple and easy-to-use interface for working with LevelDB. It offers a comprehensive set of features for database operations, including streaming, but with a more integrated approach compared to the specialized level-ws.

README

level-ws

A basic WriteStream implementation for LevelUP

level-ws provides the most basic general-case WriteStream for LevelUP. It was extracted from the core LevelUP at version 0.18.0 but is bundled with level and similar packages as it provides a general symmetry to the ReadStream in LevelUP.

level-ws is not a high-performance WriteStream, if your benchmarking shows that your particular usage pattern and data types do not perform well with this WriteStream then you should try one of the alternative WriteStreams available for LevelUP that are optimised for different use-cases.

Alternative WriteStream packages

TODO

Usage

To use level-ws you simply need to wrap a LevelUP instance and you get a createWriteStream() method on it.

var level = require('level')
var levelws = require('level-ws')
var db = level('/path/to/db')

db = levelws(db)
db.createWriteStream() // ...

db.createWriteStream([options])

A WriteStream can be obtained by calling the createWriteStream() method. The resulting stream is a Node.js streams2 Writable which operates in objectMode, accepting objects with 'key' and 'value' pairs on its write() method.

The WriteStream will buffer writes and submit them as a batch() operations where writes occur within the same tick.

var ws = db.createWriteStream()

ws.on('error', function (err) {
  console.log('Oh my!', err)
})
ws.on('close', function () {
  console.log('Stream closed')
})

ws.write({ key: 'name', value: 'Yuri Irsenovich Kim' })
ws.write({ key: 'dob', value: '16 February 1941' })
ws.write({ key: 'spouse', value: 'Kim Young-sook' })
ws.write({ key: 'occupation', value: 'Clown' })
ws.end()

The standard write(), end(), destroy() and destroySoon() methods are implemented on the WriteStream. 'drain', 'error', 'close' and 'pipe' events are emitted.

You can specify encodings both for the whole stream and individual entries:

To set the encoding for the whole stream, provide an options object as the first parameter to createWriteStream() with 'keyEncoding' and/or 'valueEncoding'.

To set the encoding for an individual entry:

writeStream.write({
    key           : new Buffer([1, 2, 3])
  , value         : { some: 'json' }
  , keyEncoding   : 'binary'
  , valueEncoding : 'json'
})

write({ type: 'put' })

If individual write() operations are performed with a 'type' property of 'del', they will be passed on as 'del' operations to the batch.

var ws = db.createWriteStream()

ws.on('error', function (err) {
  console.log('Oh my!', err)
})
ws.on('close', function () {
  console.log('Stream closed')
})

ws.write({ type: 'del', key: 'name' })
ws.write({ type: 'del', key: 'dob' })
ws.write({ type: 'put', key: 'spouse' })
ws.write({ type: 'del', key: 'occupation' })
ws.end()

db.createWriteStream({ type: 'del' })

If the WriteStream is created with a 'type' option of 'del', all write() operations will be interpreted as 'del', unless explicitly specified as 'put'.

var ws = db.createWriteStream({ type: 'del' })

ws.on('error', function (err) {
  console.log('Oh my!', err)
})
ws.on('close', function () {
  console.log('Stream closed')
})

ws.write({ key: 'name' })
ws.write({ key: 'dob' })
// but it can be overridden
ws.write({ type: 'put', key: 'spouse', value: 'Ri Sol-ju' })
ws.write({ key: 'occupation' })
ws.end()

Contributors

level-ws is only possible due to the excellent work of the following contributors:

Rod Vagg	GitHub/rvagg	Twitter/@rvagg
John Chesley	GitHub/chesles	Twitter/@chesles
Jake Verbaten	GitHub/raynos	Twitter/@raynos2
Dominic Tarr	GitHub/dominictarr	Twitter/@dominictarr
Max Ogden	GitHub/maxogden	Twitter/@maxogden
Lars-Magnus Skog	GitHub/ralphtheninja	Twitter/@ralphtheninja
David Björklund	GitHub/kesla	Twitter/@david_bjorklund
Julian Gruber	GitHub/juliangruber	Twitter/@juliangruber
Paolo Fragomeni	GitHub/hij1nx	Twitter/@hij1nx
Anton Whalley	GitHub/No9	Twitter/@antonwhalley
Matteo Collina	GitHub/mcollina	Twitter/@matteocollina
Pedro Teixeira	GitHub/pgte	Twitter/@pgte
James Halliday	GitHub/substack	Twitter/@substack

Licence & copyright

Copyright (c) 2012-2015 level-ws contributors (listed above).

level-ws is licensed under an MIT +no-false-attribs license. All rights not explicitly granted in the MIT license are reserved. See the included LICENSE file for more details.

Changelog

[0.1.0] - 2017-04-07

Changed

• Upgrade readable-stream from ~2.0.6 to ^2.2.8 (@mcollina)
• Upgrade xtend from ~2.2.1 to ^4.0.0 (@mcollina)