Security News
NIST Misses 2024 Deadline to Clear NVD Backlog
NIST has failed to meet its self-imposed deadline of clearing the NVD's backlog by the end of the fiscal year. Meanwhile, CVE's awaiting analysis have increased by 33% since June.
stream-transform
Advanced tools
Object transformations implementing the Node.js `stream.Transform` API
The stream-transform package is a simple and powerful tool for transforming data in Node.js streams. It allows you to easily modify data as it passes through a stream, making it highly useful for tasks such as data manipulation, filtering, and aggregation in real-time data processing applications.
Synchronous Transformation
This feature allows for synchronous data transformation within a stream. The provided code sample demonstrates how to read data from a CSV file, transform each record by joining its elements with a space, and then output the transformed data.
const transform = require('stream-transform');
const transformer = transform(function(record, callback){
setTimeout(function(){
callback(null, record.join(' ') + '\n');
}, 500);
}, {parallel: 5});
require('fs').createReadStream('/path/to/source.csv').pipe(transformer).pipe(process.stdout);
Asynchronous Transformation
This feature supports asynchronous transformation of stream data. The code sample shows how to asynchronously convert each piece of data to uppercase before passing it through the stream.
const transform = require('stream-transform');
const transformer = transform(function(record, callback){
process.nextTick(function(){
callback(null, record.toUpperCase());
});
});
require('fs').createReadStream('/path/to/source.txt').pipe(transformer).pipe(process.stdout);
Through2 is a tiny wrapper around Node.js streams.Transform that makes it easier to create transform streams. It is similar to stream-transform in providing a simple API for data transformation but differs in its implementation and customization options.
Event-stream is a toolkit for working with Node.js streams in a functional programming style, offering a variety of utilities for creating, combining, and consuming streams. While it provides more general stream manipulation capabilities, it shares the ability to transform stream data with stream-transform.
This project provide a simple object transformation framework implementing the
Node.js stream.Transform
API. Despite being developed as a part of the Node.js
CSV module (npm install csv
), it could be used indepently.
Documentation for the stream-transform package is available here.
Installation command is npm install stream-transform
.
You may run this script with the command node samples/sync.js
. Note how the
use
callback is only requesting one argument, the data to transform.
var transform = require('stream-transform');
transform()
.parallel(20)
.use(function(data){
data.push(data.shift())
})
.write( ['1','2','3','4'] )
.write( ['a','b','c','d'] )
.pipe process.stdout
// Output:
// 2,3,4,1
// b,c,d,a
You may run this script with the command node samples/async.js
. The call
parallel(1)
will restrict the number of parallel callback to 1, thus enabling
sequential mode. Also, note how the use
callback is requesting two arguments,
the data to transform and the callback to call when ready.
var transform = require('stream-transform');
transform()
.parallel(1)
.use(function(row, callback){
setImmediate(function(){
row.unshift(row.pop());
callback(row);
}
})
.write( ['1','2','3','4'] )
.write( ['a','b','c','d'] )
.pipe process.stdout
// Output:
// 2,3,4,1
// b,c,d,a
Most of the generator is imported from its parent project CSV in a effort to split it between the generator, the parser, the transformer and the stringifier.
Tests are executed with mocha. To install it, simple run npm install
followed by npm test
. It will install mocha and its dependencies in your
project "node_modules" directory and run the test suite. The tests run
against the CoffeeScript source files.
To generate the JavaScript files, run make build
.
The test suite is run online with Travis against the versions 0.9, 0.10 and 0.11 of Node.js.
David Worms: <https://github.com/wdavidw>
FAQs
Object transformations implementing the Node.js `stream.Transform` API
The npm package stream-transform receives a total of 1,163,337 weekly downloads. As such, stream-transform popularity was classified as popular.
We found that stream-transform demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
NIST has failed to meet its self-imposed deadline of clearing the NVD's backlog by the end of the fiscal year. Meanwhile, CVE's awaiting analysis have increased by 33% since June.
Security News
Cloudflare has launched a setup wizard allowing users to easily create and manage a security.txt file for vulnerability disclosure on their websites.
Security News
The Socket Research team breaks down a malicious npm package targeting the legitimate DOMPurify library. It uses obfuscated code to hide that it is exfiltrating browser and crypto wallet data.