streaming-json-stringify

What is streaming-json-stringify?

The 'streaming-json-stringify' npm package is designed to convert JavaScript objects into JSON strings in a memory-efficient manner by streaming the output. This is particularly useful for handling large datasets that cannot be held entirely in memory.

What are streaming-json-stringify's main functionalities?

Basic Usage

This code demonstrates the basic usage of the 'streaming-json-stringify' package. It reads an array of objects and writes them to a JSON file in a streaming manner, which is memory-efficient.

const stringify = require('streaming-json-stringify');
const fs = require('fs');

const data = [
  { id: 1, name: 'Alice' },
  { id: 2, name: 'Bob' },
  { id: 3, name: 'Charlie' }
];

const readStream = require('stream').Readable({ objectMode: true });
readStream._read = () => {};
data.forEach(item => readStream.push(item));
readStream.push(null);

const writeStream = fs.createWriteStream('output.json');
readStream.pipe(stringify()).pipe(writeStream);

Custom Replacer Function

This example shows how to use a custom replacer function with 'streaming-json-stringify'. The replacer function omits the 'age' property from the JSON output.

const stringify = require('streaming-json-stringify');
const fs = require('fs');

const data = [
  { id: 1, name: 'Alice', age: 25 },
  { id: 2, name: 'Bob', age: 30 },
  { id: 3, name: 'Charlie', age: 35 }
];

const readStream = require('stream').Readable({ objectMode: true });
readStream._read = () => {};
data.forEach(item => readStream.push(item));
readStream.push(null);

const replacer = (key, value) => {
  if (key === 'age') return undefined;
  return value;
};

const writeStream = fs.createWriteStream('output.json');
readStream.pipe(stringify(replacer)).pipe(writeStream);

Custom Space for Pretty Printing

This code demonstrates how to use the 'streaming-json-stringify' package to pretty-print JSON output with a custom space parameter for indentation.

const stringify = require('streaming-json-stringify');
const fs = require('fs');

const data = [
  { id: 1, name: 'Alice' },
  { id: 2, name: 'Bob' },
  { id: 3, name: 'Charlie' }
];

const readStream = require('stream').Readable({ objectMode: true });
readStream._read = () => {};
data.forEach(item => readStream.push(item));
readStream.push(null);

const writeStream = fs.createWriteStream('output.json');
readStream.pipe(stringify(null, 2)).pipe(writeStream);

Other packages similar to streaming-json-stringify

JSONStream

JSONStream is another package that allows for streaming JSON parsing and stringifying. It is highly flexible and can handle large JSON datasets efficiently. Compared to 'streaming-json-stringify', JSONStream offers more granular control over the JSON parsing and stringifying process, but it may be more complex to use for simple tasks.

ndjson

The 'ndjson' package is designed for newline-delimited JSON streaming. It is particularly useful for processing large datasets where each JSON object is separated by a newline. While 'ndjson' is excellent for line-by-line JSON processing, 'streaming-json-stringify' is more suited for converting entire objects or arrays into JSON strings.

oboe

Oboe.js is a library for loading JSON objects incrementally from a stream. It allows for real-time processing of JSON data as it is received. Compared to 'streaming-json-stringify', Oboe.js is more focused on parsing and handling incoming JSON data streams rather than stringifying outgoing data.

README

Streaming JSON.stringify()

Similar to JSONStream.stringify() except it is, by default, a binary stream, and it is a streams2 implementation.

Example

The main use case for this is to stream a database query to a web client. This is meant to be used only with arrays, not objects.

var Stringify = require('json-array-stream')

app.get('/things', function (req, res, next) {
  res.setHeader('Content-Type', 'application/json; charset=utf-8')

  db.things.find()
  .stream()
  .pipe(Stringify())
  .pipe(res)
})

will yield something like

[
{"_id":"123412341234123412341234"}
,
{"_id":"123412341234123412341234"}
]

Separators

• The stream always starts with '[\n'.
• Documents are separated by '\n,\n'.
• The stream is terminated with '\n]\n'.

API

Stringify([options])

Returns a Transform stream. The options are passed to the Transform constructor.

JSON.stringify options

You can override these:

var stringify = Stringify()
stringify.replacer = function () {}
stringify.space = 2

License

The MIT License (MIT)

Copyright (c) 2013 Jonathan Ong [e-mail]

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.