@discoveryjs/json-ext

What is @discoveryjs/json-ext?

The @discoveryjs/json-ext npm package provides utilities for working with JSON data, including features for streaming, parsing, and stringifying large JSON objects and arrays. It is designed to handle JSON data efficiently, making it easier to work with large JSON files or streams of JSON data.

What are @discoveryjs/json-ext's main functionalities?

Streaming JSON parse

This feature allows for parsing newline-delimited JSON (NDJSON) from a stream, enabling efficient processing of large JSON files without loading them entirely into memory.

const { createReadStream } = require('fs');
const { parseNDJSON } = require('@discoveryjs/json-ext');

const stream = createReadStream('path/to/your/file.ndjson');
parseNDJSON(stream, (obj) => {
  console.log(obj);
});

Stringify JSON to stream

This feature allows for stringifying JSON objects and piping them to a writable stream, which is useful for creating NDJSON files or streaming JSON data over the network.

const { createWriteStream } = require('fs');
const { stringifyStream } = require('@discoveryjs/json-ext');

const stream = createWriteStream('path/to/your/output.ndjson');
const objects = [{ foo: 'bar' }, { baz: 'qux' }];

stringifyStream(objects).pipe(stream);

Other packages similar to @discoveryjs/json-ext

JSONStream

JSONStream offers similar streaming JSON parsing and stringifying capabilities. It allows for parsing JSON files or streams using JSONPath-like expressions. Compared to @discoveryjs/json-ext, JSONStream focuses more on the streaming aspect and might be more suitable for scenarios where JSONPath expressions are needed for selecting data.

stream-json

stream-json provides a toolkit for processing JSON as a stream. It includes a variety of stream components for parsing, filtering, and transforming JSON data. While it shares the streaming JSON processing capability with @discoveryjs/json-ext, stream-json offers a more modular approach, allowing users to build custom processing pipelines.

README

json-ext

A set of utilities that extend the use of JSON.

Features:

• Value info: info()
• Stringify stream: stringifyStream()
• TBD Parse stream
• TBD Support for circular references
• TBD Binary representation

Install

npm install json-ext

API

stringifyStream(value[, replacer[, space]])

Works the same as JSON.stringify(), but returns an instance of ReadableStream instead of string.

Extensions:

• Output null to the output when JSON.stringify() returns undefined (since streams may not to emit undefined)
• A promise is resolving and the result value is stringifying as a regular value
• A Stream in non-object mode is piping to output as is
• A Stream in object mode is piping to output as an array of objects

Comparison with other solutions (benchmark)

info(value[, replacer[, space[, options]]])

value, replacer and space arguments are the same as for JSON.stringify().

Result is an object:

{
    minLength: Number,  // mininmal bytes when values is stringified
    circular: [...],    // list of circular references
    duplicate: [...],   // list of objects that occur more than once
    async: [...]        // list of async values, i.e. promises and streams
}

Options:

async

Type: Boolean
Default: false

Collect async values (promises and streams) or not.

continueOnCircular

Type: Boolean
Default: false

Stop walking through a value or not whenever circular reference is found. Setting option to true allows to find all circular references.

License

MIT

Changelog

0.1.0 (2020-09-08)

• Initial release