ts-iterator-utils

Typed Iterators

An API for dealing with asynchronous iterators

Installing

npm install ts-iterator-utils

Usage

Typed iterators extend the AsyncIterable interface, which makes it possible to use them in for loops:

for await (const item of _iterator_) { ... }

They also have a rich stream-like API, and are easier to use. However, there is one limitation, which is that those iterators are one-off: you can't restart the iterator. In particular, if the for loop above runs to completion, the iterator becomes empty.

API

Creating iterators

This is usually done by applying the iterator function, exported by default by this library:

const iterator = require('ts-iterator-utils').default

const myIterator1 = iterator([1,2,3])

or, in Typescript:

import iterator from 'ts-iterator-utils'

const myIterator1 = iterator([1,2,3])

The argument could be anything iterable (asynchronously or not), or a generator that produces that iterable:

const generator() = async * () => {
  yield await getFirstChunk()
  yield await getSecondChunk()
  yield await getThirdChunk()
}

const myIterator2 = iterator(generator)
const myIterator3 = iterator(generator())

Here myIterator2 and myIterator3 are equivalent.

There are a number of helper functions in the library:

1. empty(). Returns an empty iterator that does not produce any items.
2. fromPromise(pr). Here pr is a promise that resolves to an iterator. The result is simply an iterator. It's safe since you use this iterator in a for await loop anyway.
3. iterate(initial, step). Returns an iterator that yields first initial, then step(initial), then step(step(initial)) and so on. It stops when step returns undefined.
4. asyncIterate — same thing as above, but step should return a promise.
5. cycle(list). Repeats all of the list's items infinitely, over and over again.
6. unfold(initial, advance). A Swiss army knife of iterators. It returns an iterator that keeps internal state, and produces results based on that, at the same time modifying the state. Basically, it goes through the sequence of states state0, state1, state2, ..., starting with state0 = initial. Each time advance(state) function should return an object with two fields: value, which is yielded, and next, which is the next state. So, advance(state0) = {value: value0, next: state1}, advance(state1) = {value: value1, next: state2} etc., with value0, value1, ... being the iterator's output. It stops when advance returns undefined
7. asyncUnfold — same thing as above, but advance should return a promise.

Modifying iterators

Typed iterators have a rich API, resembling that of arrays or streams. Here is the summary.

map(f).

Applies the function f to all yielded values. This function has an asynchronous variant asyncMap, expecting f to return a promise instead of a result.

Example

for await (const item of iterator([1,2,3]).map((n) => n+10)) {console.log(item)}
# 11
# 12
# 13

flatMap(f)

Applies the function f to all yielded values, each time producing another iterator. It then yields all the values in all those iterators one by one. This function has an asynchronous variant asyncFlatMap, expecting f to return a promise instead of an iterator

Example

for await(const item of iterator([10,20]).flatMap((n) => iterator([n+1,n+2]))) {console.log(item)}
# 11
# 12
# 21
# 22

filter(p)

Applies the function p to all yielded values, and only keeps those that p turns into true. This function has an asynchronous variant asyncFilter, expecting p to return a promise instead of a boolean result.

Example

for await (const item of iterator([1,2,3]).filter((n) => n % 2 == 1)) {console.log(item)}
# 1
# 3

concat(otherIterator)

Yield all values of this iterator, and then follow up by all values of otherIterator.

Example

for await (const item of iterator([1,2]).concat(iterator([3,4]))) {console.log(item)}
# 1
# 2
# 3
# 4

take(n)

Restricts the iterator to its first n values.

Example

for await (const item of iterator([0,1,2]).take(2)) {console.log(item)}
# 0
# 1

drop(n)

Drops the first n values from the iterator.

Example

for await (const item of iterator([0,1,2,3]).drop(2)) {console.log(item)}
# 2
# 3

dropLast(n)

Drops the last n values from the iterator.

Example

for await (const item of iterator([0,1,2]).dropLast(2)) {console.log(item)}
# 0

batch(n)

Outputs the same values, but in batches by n in each. The last batch might be shorter than n, but is never empty.

Example

for await (const item of iterator([1,2,3,4,5]).batch(2)) {console.log(item)}
# [1,2]
# [3,4]
# [5]

groupBy(cmp)

Outputs the same values, but in batches. For any two consequetive items in the same batch cmp results in true; batches are broken up where it results in false.

Example

function cmp(a: number, b: number) {
  return a % 2 == b % 2
}
for await (const item of iterator([1,3,5,2,4,7]).groupBy(cmp)) {console.log(item)}
# [1,3,5]
# [2,4]
# [7]

scan(initial, step)

Outputs initial, and then applies step to the previously output value and the value from our iterator to get the next value. This function has an asynchronous variant asyncScan, expecting step to return a promise instead of a value.

Example

for await (const item of iterator([1,2,3,4]).scan(0, (n,m) => n + m)) {console.log(item)}
# 0
# 1
# 3
# 6
# 10

takeWhile(p)

Stops yielding values as soon as p turns false. Original iterator retains those values and would output them if required. This function has an asynchronous variant asyncTakeWhile, expecting p to return a promise instead of a boolean result.

Example

for await (const item of iterator([1,3,6,10]).takeWhile((n) => n < 5)) {console.log(item)}
# 1
# 3

dropWhile(p)

Drops all values until p turns false. This function has an asynchronous variant asyncDropWhile, expecting p to return a promise instead of a boolean result.

Example

for await (const item of iterator([1,3,6,2,10]).dropWhile((n) => n < 5)) {console.log(item)}
# 6
# 2
# 10

zipWith(other, zipper)

Applies zipper to two arguments at once. It stops when either of the iterators runs out. The other retains all remaining values. This function has an asynchronous variant asyncZipWith, expecting zipper to return a promise instead of a result.

Example

for await (const item of iterator([1,2,3]).zipWith(iterator([10,20]), (n, m) => n + m)) {console.log(item)}
# 11
# 22

iterator.merge(iterables, compare)

This is a static function. It merges several iterators, assumed to output their values in the sorted order, into one, preserving the sortedness. compare function is used to determin which value is smaller than the other.

Example

for await (const item of iterator.merge([[1,5,6],[2,3,7]], (n, m) => n - m)) {console.log(item)}
# 1
# 2
# 3
# 5
# 6
# 7

sorted(compare, error)

Checks that the iterator is sorted; throws an exception as soon as it finds a new item which is smaller than the previous one, according to the compare function. error function is used to determine which error to throw. This is intended to be used in conjunction with merge above.

Example

const it = iterator([1,3,2,4]).sorted((n, m) => n - m, (n, m) => new Error(`${m} is smaller than ${n})`)
for await (const item of it) {console.log(item)}
# 1
# 3
# Error: 2 is smaller than 3

Exctracting data from iterators

As said previously, iterators can be used in for-await loops. Beside that, there is a bunch of functions to get data from them.

collect()

Returns a promise, resolving to an array of all items remaining in the iterator.

Example

await iterator([1,2,3]).collect()
# [1,2,3]

pop()

Returns a promise, resolving to the first element remaining in the iterator, or undefined if there is no such element. All other elements remain in the iterator and can be collected later.

Example

await iterator([1,2,3]).pop()
# 1

last()

Returns a promise, resolving to the last element in the iterator, or undefined if there is no such element. Iterator is fully emptied in the process.

Example

await iterator([1,2,3]).last()
# 3

isEmpty()

Returns a promise, resolving to true if the iterator is empty, or false if it's not done yet.

Example

await iterator([1,2,3]).isEmpty()

# false

fold(initial, step)

Applies step to initial and all values in the iterator in turn; basically, if v0, v1 and v2 are the values in the iterator, it would compute step(step(step(initial, v0), v1), v2). However, if step returns undefined, then the previous value is returned and the rest of the values (starting with the one that made step fail) are retained in the iterator. This function has an asynchronous variant asyncFold, expecting step to return a promise instead of a result.

Example

await iterator([1,2,3]).fold(0, (n, m) => n + m)
# 6

pipe(writable)

Makes the iterator empty itself into the Writable stream. An error is raised if writing is, for some reason, impossible (e.g., if saving an object into a stream with objectMode=false).

Example

import fs from 'fs'
await iterator(['abc', 'def', 'xyz']).pipe(fs.createWriteStream('temp.txt'))
# temp.txt now contains "abcdefxyz"