jsonlines

What is jsonlines?

The jsonlines npm package is used for reading and writing JSON Lines, a format for storing structured data that is easy to process one record at a time. Each line in a JSON Lines file is a valid JSON value, separated by newline characters.

What are jsonlines's main functionalities?

Reading JSON Lines

This code demonstrates how to read a JSON Lines file using the jsonlines package. It creates a readable stream from an input file and pipes it to the jsonlines parser. The parser emits 'data' events for each parsed JSON object.

const jsonlines = require('jsonlines');
const fs = require('fs');

const input = fs.createReadStream('input.jsonl');
const reader = jsonlines.parse();

input.pipe(reader);

reader.on('data', (obj) => {
  console.log('Parsed object:', obj);
});

reader.on('end', () => {
  console.log('Finished reading file.');
});

Writing JSON Lines

This code demonstrates how to write JSON objects to a JSON Lines file using the jsonlines package. It creates a writable stream to an output file and pipes it to the jsonlines stringifier. JSON objects are written to the stringifier, which then writes them to the file.

const jsonlines = require('jsonlines');
const fs = require('fs');

const output = fs.createWriteStream('output.jsonl');
const writer = jsonlines.stringify();

writer.pipe(output);

writer.write({ foo: 'bar' });
writer.write({ baz: 'qux' });
writer.end();

Other packages similar to jsonlines

ndjson

The ndjson package is another library for working with Newline Delimited JSON (NDJSON) files. It provides similar functionality to jsonlines, including parsing and stringifying NDJSON data. The main difference is in the API design and the specific features offered by each package.

json-stream

The json-stream package is designed for streaming JSON data. It can parse JSON objects from a stream and emit events for each object. While it is not specifically designed for JSON Lines, it can be used to process JSON Lines data by handling each line as a separate JSON object.

stream-json

The stream-json package is a comprehensive library for processing JSON data in a streaming fashion. It can handle large JSON files and streams, and it provides tools for parsing, stringifying, and transforming JSON data. It is more versatile than jsonlines but also more complex to use.

README

JSONLines for Node.js

Parse JSONLines with Node.js.

Installation

npm install --save jsonlines

Usage

var jsonlines = require('jsonlines')
var parser = jsonlines.parse()

parser.on('data', function (data) {
  console.log('Got json:', data)
})

parser.on('end', function () {
  console.log('No more data')
})

parser.write('{ "test": "This is a test!" }\n')
parser.write('{ "jsonlines": "is awesome" }')
parser.end()

var jsonlines = require('jsonlines')
var stringifier = jsonlines.stringify()

stringifier.pipe(process.stdout)

stringifier.write({ test: 'This is a test!' })
stringifier.write({ jsonlines: 'is awesome' })
stringifier.end()

API

.parse([options])

Returns a transform stream that turns newline separated json into a stream of javascript values.

options is an optional object with the keys documented below.

.stringify()

Returns a transform stream that turns javascript values into a stream of newline separated json.

Options

emitInvalidLine

If true, instead of emitting an error and cancelling the stream when an invalid line is proccessed, an invalid-line event is emitted with the same error. This is very useful when processing text that have mixed plain text and json data.

Example:

var jsonlines = require('jsonlines')
var parser = jsonlines.parse({ emitInvalidLines: true })

parser.on('data', function (data) {
  console.log('Got json:', data)
})

parser.on('invalid-line', function (err) {
  console.log('Got text:', err.source)
})

parser.write('{ "test": "This is a test!" }\n')
parser.write('This is some plain text\n')
parser.write('{ "jsonlines": "is awesome" }')
parser.end()

Output:

Got json: { test: 'This is a test!' }
Got text: This is some plain text
Got json: { jsonlines: 'is awesome' }