thread-stream

What is thread-stream?

The thread-stream npm package is designed to facilitate the creation of writable streams that operate within worker threads in Node.js. This allows for offloading CPU-intensive tasks or I/O-bound tasks to separate threads, improving the performance and responsiveness of Node.js applications.

What are thread-stream's main functionalities?

Creating a basic thread stream

This code demonstrates how to create a basic thread stream using the thread-stream package. It involves specifying a worker file and optional worker data. The stream created can be used like any other writable stream in Node.js.

const ThreadStream = require('thread-stream');
const stream = new ThreadStream({
  filename: path.join(__dirname, 'worker.js'),
  workerData: { exampleData: 'data' },
  sync: false
});
stream.write('Hello, world!');
stream.end();

Handling stream events

This example shows how to handle data and end events from a thread stream. It allows processing of data received from the worker thread and handling the end of the stream.

stream.on('data', (data) => {
  console.log('Data from stream:', data);
});
stream.on('end', () => {
  console.log('Stream ended.');
});

Other packages similar to thread-stream

workerpool

Workerpool is a package that manages a pool of workers and handles the execution of tasks in these workers. It is similar to thread-stream in that it helps offload tasks to separate threads but differs in its approach by managing multiple workers and tasks rather than focusing on stream-based communication.

bthreads

Bthreads provides a wrapper around Node.js worker threads, simplifying thread management and communication. While it shares the goal of enhancing performance through threads like thread-stream, bthreads does not specifically focus on stream-based interfaces, offering a broader range of thread management features.

README

thread-stream

A streaming way to send data to a Node.js Worker Thread.

install

npm i thread-stream

Usage

'use strict'

const ThreadStream = require('thread-stream')
const { join } = require('path')

const stream = new ThreadStream({
  filename: join(__dirname, 'worker.js'),
  workerData: { dest },
  workerOpts: {}, // Other options to be passed to Worker
  sync: false, // default
})

stream.write('hello')

// Asynchronous flushing
stream.flush(function () {
  stream.write(' ')
  stream.write('world')

  // Synchronous flushing
  stream.flushSync()
  stream.end()
})

In worker.js:

'use strict'

const fs = require('fs')
const { once } = require('events')

async function run (opts) {
  const stream = fs.createWriteStream(opts.dest)
  await once(stream, 'open')
  return stream
}

module.exports = run

Make sure that the stream emits 'close' when the stream completes. This can usually be achieved by passing the autoDestroy: true flag your stream classes.

License

MIT