level-ws

What is level-ws?

The level-ws npm package is a WriteStream implementation for LevelDB, which allows for efficient batch writing of data to a LevelDB database. It is particularly useful for handling large amounts of data that need to be written to the database in a streaming fashion.

What are level-ws's main functionalities?

Batch Writing

This feature allows you to write multiple key-value pairs to a LevelDB database in a batch. The code sample demonstrates how to create a WriteStream and write multiple entries to the database.

const level = require('level');
const levelws = require('level-ws');

const db = level('./mydb');
const ws = levelws(db);

ws.write({ key: 'name', value: 'Alice' });
ws.write({ key: 'age', value: '30' });
ws.end();

Stream Handling

This feature allows you to handle streams of data and pipe them directly into the LevelDB database. The code sample demonstrates how to create a readable stream and pipe it into the WriteStream.

const level = require('level');
const levelws = require('level-ws');
const { Readable } = require('stream');

const db = level('./mydb');
const ws = levelws(db);

const data = [
  { key: 'name', value: 'Alice' },
  { key: 'age', value: '30' }
];

const readable = Readable.from(data);
readable.pipe(ws);

Other packages similar to level-ws

levelup

LevelUP is a Node.js module that provides a higher-level API for interacting with LevelDB. It offers a more user-friendly interface compared to the lower-level LevelDB bindings. While level-ws focuses on WriteStream functionality, LevelUP provides a broader range of database operations including read, write, delete, and batch operations.

leveldown

LevelDOWN is the backend store for LevelDB in Node.js. It provides the low-level bindings to the LevelDB library. While level-ws is specifically for streaming writes, LevelDOWN offers the foundational operations for interacting with LevelDB, such as put, get, and del.

level

The level package is a convenience package that bundles LevelUP and LevelDOWN, providing a simple and easy-to-use interface for working with LevelDB. It offers a comprehensive set of features for database operations, including streaming, but with a more integrated approach compared to the specialized level-ws.

README

level-ws

A basic WriteStream implementation for levelup

level-ws provides the most basic general-case WriteStream for levelup. It was extracted from the core levelup at version 0.18.0.

level-ws is not a high-performance WriteStream. If your benchmarking shows that your particular usage pattern and data types do not perform well with this WriteStream then you should try one of the alternative WriteStreams available for levelup that are optimised for different use-cases.

If you are upgrading: please see UPGRADING.md.

Usage

var level = require('level')
var WriteStream = require('level-ws')

var db = level('/path/to/db')
var ws = WriteStream(db) // ...

API

ws = WriteStream(db[, options])

Creates a Writable stream which operates in objectMode, accepting objects with 'key' and 'value' pairs on its write() method.

The optional options argument may contain:

• type (string, default: 'put'): Default batch operation for missing type property during ws.write().

The WriteStream will buffer writes and submit them as a batch() operations where writes occur within the same tick.

var ws = WriteStream(db)

ws.on('error', function (err) {
  console.log('Oh my!', err)
})
ws.on('close', function () {
  console.log('Stream closed')
})

ws.write({ key: 'name', value: 'Yuri Irsenovich Kim' })
ws.write({ key: 'dob', value: '16 February 1941' })
ws.write({ key: 'spouse', value: 'Kim Young-sook' })
ws.write({ key: 'occupation', value: 'Clown' })
ws.end()

The standard write(), end() and destroy() methods are implemented on the WriteStream. 'drain', 'error', 'close' and 'pipe' events are emitted.

You can specify encodings for individual entries by setting .keyEncoding and/or .valueEncoding:

writeStream.write({
  key: new Buffer([1, 2, 3]),
  value: { some: 'json' },
  keyEncoding: 'binary',
  valueEncoding : 'json'
})

If individual write() operations are performed with a 'type' property of 'del', they will be passed on as 'del' operations to the batch.

var ws = WriteStream(db)

ws.on('error', function (err) {
  console.log('Oh my!', err)
})
ws.on('close', function () {
  console.log('Stream closed')
})

ws.write({ type: 'del', key: 'name' })
ws.write({ type: 'del', key: 'dob' })
ws.write({ type: 'put', key: 'spouse' })
ws.write({ type: 'del', key: 'occupation' })
ws.end()

If the WriteStream is created with a 'type' option of 'del', all write() operations will be interpreted as 'del', unless explicitly specified as 'put'.

var ws = WriteStream(db, { type: 'del' })

ws.on('error', function (err) {
  console.log('Oh my!', err)
})
ws.on('close', function () {
  console.log('Stream closed')
})

ws.write({ key: 'name' })
ws.write({ key: 'dob' })
// but it can be overridden
ws.write({ type: 'put', key: 'spouse', value: 'Ri Sol-ju' })
ws.write({ key: 'occupation' })
ws.end()

Contributing

Level/level-ws is an OPEN Open Source Project. This means that:

Individuals making significant and valuable contributions are given commit-access to the project to contribute as they see fit. This project is more like an open wiki than a standard guarded open source project.

See the Contribution Guide for more details.

Donate

Support us with a monthly donation on Open Collective and help us continue our work. Your logo or avatar will be displayed on our 28+ GitHub repositories and npm packages. 💖

Active financial contributors

Past financial contributors

License

MIT

Changelog

[3.0.0] - 2021-04-25

If you are upgrading: please see UPGRADING.md.

Changed

• Breaking: drop node 6 and 8 (Level/community#98) (2b04c28) (Vincent Weevers)
• Breaking: modernize syntax (Level/community#98) (2319634, 3fc0829) (Vincent Weevers)