json-stream-stringify

What is json-stream-stringify?

The json-stream-stringify npm package is designed to convert JavaScript objects into JSON strings in a memory-efficient manner by streaming the output. This is particularly useful for handling large objects that might otherwise consume too much memory if converted to JSON all at once.

What are json-stream-stringify's main functionalities?

Basic Usage

This feature demonstrates the basic usage of the json-stream-stringify package. It shows how to create a stream from a large JavaScript object and pipe it to a file, thereby converting the object to a JSON string in a memory-efficient way.

const JSONStreamStringify = require('json-stream-stringify');
const fs = require('fs');

const largeObject = { /* large object data */ };
const jsonStringifyStream = new JSONStreamStringify(largeObject);

jsonStringifyStream.pipe(fs.createWriteStream('output.json'));

Custom Replacer Function

This feature demonstrates how to use a custom replacer function with json-stream-stringify. The replacer function can be used to filter or modify the values being stringified. In this example, all string properties are excluded from the output.

const JSONStreamStringify = require('json-stream-stringify');
const fs = require('fs');

const largeObject = { /* large object data */ };
const replacer = (key, value) => {
  if (typeof value === 'string') {
    return undefined; // Exclude all string properties
  }
  return value;
};
const jsonStringifyStream = new JSONStreamStringify(largeObject, replacer);

jsonStringifyStream.pipe(fs.createWriteStream('output.json'));

Custom Space Argument

This feature demonstrates how to use the space argument to format the JSON output with indentation. In this example, the JSON output will be pretty-printed with an indentation of 2 spaces.

const JSONStreamStringify = require('json-stream-stringify');
const fs = require('fs');

const largeObject = { /* large object data */ };
const jsonStringifyStream = new JSONStreamStringify(largeObject, null, 2);

jsonStringifyStream.pipe(fs.createWriteStream('output.json'));

Other packages similar to json-stream-stringify

JSONStream

JSONStream is a package that provides streaming JSON parsing and stringifying. It is similar to json-stream-stringify in that it allows for memory-efficient handling of large JSON objects. However, JSONStream offers more flexibility with its ability to parse JSON streams in addition to stringifying them.

stream-json

stream-json is another package that provides tools for working with JSON in a streaming fashion. It includes parsers, stringifiers, and utilities for processing JSON data in a memory-efficient way. Compared to json-stream-stringify, stream-json offers a more modular approach with separate components for different tasks.

oboe

oboe is a library for working with JSON in a streaming manner. It allows for both parsing and stringifying JSON data. Oboe is particularly useful for handling large JSON responses from HTTP requests. Compared to json-stream-stringify, oboe provides more features for working with JSON data in real-time.

README

JSON Stream Stringify

JSON Stringify as a Readable Stream with rescursive resolving of any readable streams and Promises.

Main Features

• Promises are rescursively resolved and the result is piped through JSONStreamStreamify
• Streams (ObjectMode) are piped through a transform which pipes the data through JSONStreamStreamify (enabling recursive resolving)
• Streams (Non-ObjectMode) is stringified and piped
• Output is streamed optimally with as small chunks as possible
• Great memory management with reference release post process (When a key and value has been processed the value is dereferenced)
• Stream pressure handling

Install

npm install --save json-stream-stringify

API

JSONStreamStringify(value[, replacer])

Convert value to JSON string. Returns a readable stream.

• value Any data to convert to JSON.
• replacer Optional Function(key, value) or Array.
  As a function the returned value replaces the value associated with the key. Details
  As an array all other keys are filtered. Details

Example Usage

const JSONStreamStreamify = require('json-stream-stringify');

JSONStreamStreamify({
    aPromise: Promise.resolve(Promise.resolve("text")), // Promise may resolve more promises and streams which will be consumed and resolved
    aStream: ReadableObjectStream({a:1}, 'str'), // Stream may write more streams and promises which will be consumed and resolved
    arr: [1, 2, Promise.resolve(3), Promise.resolve([4, 5]), ReadableStream('a', 'b', 'c')],
    date: new Date(2016, 0, 2)
}).pipe(process.stdout);

Output (each line represents a write from JSONStreamStreamify)

{
"aPromise":
"text"
"aStream":
[
{
"a":
1
}
,
"str"
]
"arr":
[
1
,
2
,
3
,
[
4
,
5
]
,
"
a
b
c
"
],
"date":
"2016-01-01T23:00:00.000Z"
}

Practical Example with Express + Mongoose

app.get('/api/users', (req, res, next) => JSONStreamStreamify(Users.find().stream()).pipe(res));

TODO

• Space option
• Circular dependency detection/handling (infinite loops may occur as it is)

Feel free to contribute.

Technical Notes

Uses toJSON when available, and JSON.stringify to stringify everything but objects and arrays.
Streams with ObjectMode=true are output as arrays while ObjectMode=false output as a concatinated string (each chunk is piped with transforms).

Requirements

NodeJS >4.2.2

License

MIT

Copyright (c) 2016 Faleij [e-mail]