streaming-json-stringify

What is streaming-json-stringify?

The 'streaming-json-stringify' npm package is designed to convert JavaScript objects into JSON strings in a memory-efficient manner by streaming the output. This is particularly useful for handling large datasets that cannot be held entirely in memory.

What are streaming-json-stringify's main functionalities?

Basic Usage

This code demonstrates the basic usage of the 'streaming-json-stringify' package. It reads an array of objects and writes them to a JSON file in a streaming manner, which is memory-efficient.

const stringify = require('streaming-json-stringify');
const fs = require('fs');

const data = [
  { id: 1, name: 'Alice' },
  { id: 2, name: 'Bob' },
  { id: 3, name: 'Charlie' }
];

const readStream = require('stream').Readable({ objectMode: true });
readStream._read = () => {};
data.forEach(item => readStream.push(item));
readStream.push(null);

const writeStream = fs.createWriteStream('output.json');
readStream.pipe(stringify()).pipe(writeStream);

Custom Replacer Function

This example shows how to use a custom replacer function with 'streaming-json-stringify'. The replacer function omits the 'age' property from the JSON output.

const stringify = require('streaming-json-stringify');
const fs = require('fs');

const data = [
  { id: 1, name: 'Alice', age: 25 },
  { id: 2, name: 'Bob', age: 30 },
  { id: 3, name: 'Charlie', age: 35 }
];

const readStream = require('stream').Readable({ objectMode: true });
readStream._read = () => {};
data.forEach(item => readStream.push(item));
readStream.push(null);

const replacer = (key, value) => {
  if (key === 'age') return undefined;
  return value;
};

const writeStream = fs.createWriteStream('output.json');
readStream.pipe(stringify(replacer)).pipe(writeStream);

Custom Space for Pretty Printing

This code demonstrates how to use the 'streaming-json-stringify' package to pretty-print JSON output with a custom space parameter for indentation.

const stringify = require('streaming-json-stringify');
const fs = require('fs');

const data = [
  { id: 1, name: 'Alice' },
  { id: 2, name: 'Bob' },
  { id: 3, name: 'Charlie' }
];

const readStream = require('stream').Readable({ objectMode: true });
readStream._read = () => {};
data.forEach(item => readStream.push(item));
readStream.push(null);

const writeStream = fs.createWriteStream('output.json');
readStream.pipe(stringify(null, 2)).pipe(writeStream);

Other packages similar to streaming-json-stringify

JSONStream

JSONStream is another package that allows for streaming JSON parsing and stringifying. It is highly flexible and can handle large JSON datasets efficiently. Compared to 'streaming-json-stringify', JSONStream offers more granular control over the JSON parsing and stringifying process, but it may be more complex to use for simple tasks.

ndjson

The 'ndjson' package is designed for newline-delimited JSON streaming. It is particularly useful for processing large datasets where each JSON object is separated by a newline. While 'ndjson' is excellent for line-by-line JSON processing, 'streaming-json-stringify' is more suited for converting entire objects or arrays into JSON strings.

oboe

Oboe.js is a library for loading JSON objects incrementally from a stream. It allows for real-time processing of JSON data as it is received. Compared to 'streaming-json-stringify', Oboe.js is more focused on parsing and handling incoming JSON data streams rather than stringifying outgoing data.

README

Streaming JSON.stringify()

Similar to JSONStream.stringify() except it is, by default, a binary stream, and it is a streams2 implementation.

Example

The main use case for this is to stream a database query to a web client. This is meant to be used only with arrays, not objects.

var Stringify = require('streaming-json-stringify')

app.get('/things', function (req, res, next) {
  res.setHeader('Content-Type', 'application/json; charset=utf-8')

  db.things.find()
  .stream()
  .pipe(Stringify())
  .pipe(res)
})

will yield something like

[
{"_id":"123412341234123412341234"}
,
{"_id":"123412341234123412341234"}
]

Separators

• The stream always starts with '[\n'.
• Documents are separated by '\n,\n'.
• The stream is terminated with '\n]\n'.

Stringifier

By default, json-stringify-safe is used to convert objects into strings. This can be configured with options.stringifier.

API

Stringify([options])

Returns a Transform stream. The options are passed to the Transform constructor.

JSON.stringify options

You can override these:

var stringify = Stringify()
stringify.replacer = function () {}
stringify.space = 2
stringify.opener = '['
stringify.seperator = ','
stringify.closer = ']'
stringify.stringifier = JSON.stringify