@rushstack/stream-collator

What is @rushstack/stream-collator?

@rushstack/stream-collator is a Node.js library that allows you to manage and collate multiple streams of output, ensuring that the output from different streams is properly interleaved and displayed in a coherent manner. This is particularly useful for managing the output of concurrent processes, such as build tools or test runners.

What are @rushstack/stream-collator's main functionalities?

Collating Multiple Streams

This feature allows you to register multiple streams and collate their output. The collator ensures that the output from different streams is properly interleaved and displayed in a coherent manner.

const { StreamCollator } = require('@rushstack/stream-collator');

const collator = new StreamCollator();

const stream1 = collator.registerStream('stream1');
const stream2 = collator.registerStream('stream2');

stream1.write('Output from stream 1\n');
stream2.write('Output from stream 2\n');

stream1.end();
stream2.end();

collator.on('data', (data) => {
  process.stdout.write(data);
});

Handling Stream Completion

This feature allows you to handle the completion of all registered streams. The 'end' event is emitted when all streams have finished writing.

const { StreamCollator } = require('@rushstack/stream-collator');

const collator = new StreamCollator();

const stream1 = collator.registerStream('stream1');
const stream2 = collator.registerStream('stream2');

stream1.write('Output from stream 1\n');
stream2.write('Output from stream 2\n');

stream1.end();
stream2.end();

collator.on('end', () => {
  console.log('All streams have completed');
});

Error Handling

This feature allows you to handle errors from any of the registered streams. The 'error' event is emitted when an error occurs in any of the streams.

const { StreamCollator } = require('@rushstack/stream-collator');

const collator = new StreamCollator();

const stream1 = collator.registerStream('stream1');
const stream2 = collator.registerStream('stream2');

stream1.write('Output from stream 1\n');
stream2.write('Output from stream 2\n');

stream1.emit('error', new Error('Stream 1 error'));

collator.on('error', (err) => {
  console.error('Error:', err.message);
});

Other packages similar to @rushstack/stream-collator

merge-stream

The 'merge-stream' package allows you to merge multiple streams into a single stream. Unlike @rushstack/stream-collator, it does not provide advanced features for managing the interleaving of output or handling stream completion and errors in a coordinated manner.

multistream

The 'multistream' package allows you to combine multiple streams into a single stream. It focuses on sequentially combining streams rather than interleaving their output. It is simpler but less powerful compared to @rushstack/stream-collator.

stream-combiner2

The 'stream-combiner2' package allows you to combine multiple streams into a pipeline. It is useful for creating complex stream processing pipelines but does not provide the same level of control over output interleaving and stream management as @rushstack/stream-collator.

README

@rushstack/stream-collator

This library enables a tool to display live console output from multiple concurrent processes, while ensuring that their output does not get jumbled together.

How does it work?

The stream-collator manages the output of these streams, ensuring that no two streams are writing to the console at the same time. At any given time, one stream registered with the collator is the active stream, which means that particular stream will be live streaming, while the others will wait for that stream to finish before their output is displayed.

For example, if you have 3 streams (e.g. from using child_process.spawn()).

Stream A will write: AAAAA

Stream B will write: BBBBBBBBBBBBBBBBBBBB

Stream C will write: CCCCCCCCCC

If these streams are all being piped directly to stdout (without @rushstack/stream-collator), you could end up with jumbled output:

ABACCCBCCCCBBABBCBBABBBBBBCCAB

Something like the following would be much more useful to users of your application:

AAAAABBBBBBBBBBBBBBBCCCCCCCCCC

This is where the @rushstack/stream-collator comes in!

The active stream

At any given time, a single stream is designated as the active stream. The output of the active stream will always be live-streamed. This is particularly useful for long-running streams. When the active stream finishes, a new stream is selected as the active stream and all of its contents up to that point will be emitted. Whenever an active stream finishes, all background streams which have been completed will be emitted.

Usage

🚨 This is an early preview release. Please report issues! 🚨

WITH VERSION 4.X, THIS PACKAGE HAS BEEN REDESIGNED TO USE THE NEW @rushstack/terminal SYSTEM. IN THE NEXT RELEASE, THE CollatedTerminal API WILL BE REPLACED WITH THE Terminal API.

The usage instructions will be updated once that refactoring is complete.

Links

• CHANGELOG.md - Find out what's new in the latest version
• API Reference

@rushstack/stream-collator is part of the Rush Stack family of projects.