stream-transform

What is stream-transform?

The stream-transform package is a simple and powerful tool for transforming data in Node.js streams. It allows you to easily modify data as it passes through a stream, making it highly useful for tasks such as data manipulation, filtering, and aggregation in real-time data processing applications.

What are stream-transform's main functionalities?

Synchronous Transformation

This feature allows for synchronous data transformation within a stream. The provided code sample demonstrates how to read data from a CSV file, transform each record by joining its elements with a space, and then output the transformed data.

const transform = require('stream-transform');
const transformer = transform(function(record, callback){
  setTimeout(function(){
    callback(null, record.join(' ') + '\n');
  }, 500);
}, {parallel: 5});
require('fs').createReadStream('/path/to/source.csv').pipe(transformer).pipe(process.stdout);

Asynchronous Transformation

This feature supports asynchronous transformation of stream data. The code sample shows how to asynchronously convert each piece of data to uppercase before passing it through the stream.

const transform = require('stream-transform');
const transformer = transform(function(record, callback){
  process.nextTick(function(){
    callback(null, record.toUpperCase());
  });
});
require('fs').createReadStream('/path/to/source.txt').pipe(transformer).pipe(process.stdout);

Other packages similar to stream-transform

through2

Through2 is a tiny wrapper around Node.js streams.Transform that makes it easier to create transform streams. It is similar to stream-transform in providing a simple API for data transformation but differs in its implementation and customization options.

event-stream

Event-stream is a toolkit for working with Node.js streams in a functional programming style, offering a variety of utilities for creating, combining, and consuming streams. While it provides more general stream manipulation capabilities, it shares the ability to transform stream data with stream-transform.

README

Part of the CSV module, this project is a simple object transformation framework. It implements the Node.js stream.Transform API.
It also provides a simple callback-based API for convenience.
It is both extremely easy to use and powerful.

Documentation for the "csv-parse" package is available here.

Features

• Follow the Node.js streaming API
• Simplicity with the optional callback API
• Synchronous and asynchronous user handler functions
• Accepts arrays of strings, or arrays of objects as input
• Sequential or user-defined concurrent execution
• Skip and create new records
• Alter or clone input data
• BSD License

Usage

Refer to the project webpage for an exhaustive list of options and some usage examples.

The module is built on the Node.js Stream API. For the sake of simplify, a simple callback API is also provided. To give you a quick look, here's an example of the callback API:

var transform = require('stream-transform');

input = [ [ '1', '2', '3', '4' ], [ 'a', 'b', 'c', 'd' ] ];
transform(input, function(data){
  data.push(data.shift());
  return data.join(',')+'\n';
}, function(err, output){
  output.should.eql([ '2,3,4,1\n', 'b,c,d,a\n' ]);
});

Development

Tests are executed with mocha. To install it, simple run npm install followed by npm test. It will install mocha and its dependencies in your project "node_modules" directory and run the test suite. The tests run against the CoffeeScript source files.

To generate the JavaScript files, run npm run coffee.

The test suite is run online with Travis against the versions 0.10, 0.11 and 0.12 of Node.js.