buffered-async-iterable

Buffered parallel processing of async iterables / generators.

Usage

Simple

import { bufferedAsyncMap } from 'buffered-async-iterable';

async function * asyncGenerator() {
  yield ...
}

const mappedIterator = bufferedAsyncMap(asyncGenerator(), async (item) => {
  // Apply additional async lookup / processing
});

for await (const item of mappedIterator) {
  // Consume the buffered async iterable
}

Array input

import { bufferedAsyncMap } from 'buffered-async-iterable';

const mappedIterator = bufferedAsyncMap(['foo'], async (item) => {
  // Apply additional async lookup / processing
});

for await (const item of mappedIterator) {
  // Consume the buffered async iterable
}

Async generator result

import { bufferedAsyncMap } from 'buffered-async-iterable';

const mappedIterator = bufferedAsyncMap(['foo'], async function * (item) => {
  // Apply additional async lookup / processing
  yield ...
  yield * ...
});

for await (const item of mappedIterator) {
  // Consume the buffered async iterable
}

API

bufferedAsyncMap()

Iterates and applies the callback to up to bufferSize items from input yielding values as they resolve.

Syntax

bufferedAsyncMap(input, callback[, { bufferSize=6, ordered=false }]) => AsyncIterableIterator

Arguments

• input – either an async iterable, an ordinare iterable or an array
• callback(item) – should be either an async generator or an ordinary async function. Items from async generators are buffered in the main buffer and the buffer is refilled by the one that has least items in the current buffer (input is considered equal to sub iterators in this regard when refilling the buffer)

Options

• bufferSize – optional – defaults to 6, sets the max amount of simultanoeus items that processed at once in the buffer.
• ordered – optional – defaults to false, when true the result will be returned in order instead of unordered

mergeIterables()

Merges all given (async) iterables in parallel, returning the values as they resolve

Syntax

mergeIterables(input[, { bufferSize=6 }]) => AsyncIterableIterator

Arguments

• input – an array of async iterables, ordinare iterables and/or arrays

Options

• bufferSize – optional – defaults to 6, sets the max amount of simultanoeus items that processed at once in the buffer.

Similar modules

• hwp – similar module by @mcollina

Changelog

1.0.1 (2024-06-24)

🧹 Chores

• deps: update dev deps (1bf86a4)
• deps: update to neostandard based linting (b4f8b57)
• deps: update type dependencies (1fdcda2)