What is ldjson-stream?
The ldjson-stream npm package is designed to handle line-delimited JSON (LDJSON) streams. It provides utilities for parsing and stringifying LDJSON data, which is useful for processing large datasets in a memory-efficient manner.
What are ldjson-stream's main functionalities?
Parsing LDJSON
This feature allows you to parse a stream of line-delimited JSON data. The code sample reads from a file named 'data.ldjson' and parses each line as a JSON object, logging each object to the console.
const ldjson = require('ldjson-stream');
const fs = require('fs');
fs.createReadStream('data.ldjson')
.pipe(ldjson.parse())
.on('data', obj => {
console.log(obj);
});
Stringifying LDJSON
This feature allows you to stringify a stream of JSON objects into line-delimited JSON format. The code sample takes an array of objects and writes them to a file named 'output.ldjson' in LDJSON format.
const ldjson = require('ldjson-stream');
const fs = require('fs');
const data = [
{ name: 'Alice', age: 30 },
{ name: 'Bob', age: 25 }
];
const writeStream = fs.createWriteStream('output.ldjson');
const stringifyStream = ldjson.serialize();
stringifyStream.pipe(writeStream);
data.forEach(item => stringifyStream.write(item));
stringifyStream.end();
Other packages similar to ldjson-stream
JSONStream
JSONStream is a popular package for working with streaming JSON data. It provides similar functionality to ldjson-stream, including parsing and stringifying JSON data. However, JSONStream is more general-purpose and can handle various JSON formats, not just line-delimited JSON.
ndjson
ndjson is another package specifically designed for handling newline-delimited JSON streams. It offers similar features to ldjson-stream, such as parsing and stringifying NDJSON data. The main difference is in the API design and implementation details.
event-stream
event-stream is a toolkit for working with streams in Node.js. It includes utilities for parsing and stringifying JSON data, among other stream-related functionalities. While it is not focused solely on LDJSON, it can be used to achieve similar results.
ldjson-stream
streaming line delimited json parser + serializer
usage
var ldj = require('ldjson-stream')
ldj.parse()
returns a transform stream that accepts newline delimited json and emits objects
example newline delimited json:
data.txt
:
{"foo": "bar"}
{"hello": "world"}
If you want to discard non-valid JSON messages, you can call ldj.parse({strict: false})
usage:
fs.createReadStream('data.txt')
.pipe(ldj.parse())
.on('data', function(obj) {
})
ldj.serialize()
returns a transform stream that accepts json objects and emits newline delimited json
example usage:
var serialize = ldj.serialize()
serialize.on('data', function(line) {
})
serialize.write({"foo": "bar"})
serialize.end()
license
BSD