async-iterators

#async-iterators

Useful abstractions and utility functions for async iterators in Node.js.

An async iterator is an object with a next(cb) method. Invoking the method should return the next item of an underlying data source. The callback should be a function of type function(err, value). If the iterator has no more data to read, it will call the callback with value == undefined.

Async iterators can easily be created from Node.js Readable Streams by using stream-iterator.

An example with a pointless iterator that asynchronously returns the numbers from 1 to 100:

var iterators = require('async-iterators')

function createExampleIterator = function() {
  var i = 0
  return {
    next: function(cb) {
      i++
      if (i == 100) return cb(null, undefined)
      cb(null, i)
    }
  }
}

var myIterator = createExampleIterator()

// wrap myIterator with a map iterator that doubles all results
var doublingIterator = iterators.map(iterator, function(err, each) {
  return each * 2
})

// pipe the iterator to an array
iterators.toArray(doublingIterator, function(err, res) {
  console.log(res)
})

##Documentation ###Iterator Sources

• fromArray
• fromReadableStream

###Transforming Iterators

• map / mapAsync
• filter / filterAsync
• range
• buffer

###Iterator Targets

• toArray
• toWritableStream

###Utilities

• forEach

##Iterator Sources

### fromArray(array) Creates an iterator from an array.

var arrayIterator = iterators.fromArray(numbers)

### fromReadableStream(readableStream) Creates an iterator from a [Readable Stream](http://nodejs.org/api/stream.html#stream_class_stream_readable).

var readStream = fs.createReadStream('input.txt', {encoding: 'utf8'})
var streamIterator = iterators.fromReadableStream(readStream)

##Transforming Iterators

### map(iterator, mapFn) Create an iterator that applies a map function to transform each value of the source iterator.

var mapIterator = iterators.map(someNumberIterator, function(err, each) {
  return each * 2
})

// pipe the iterator to an array:
iterators.toArray(mapIterator, function(err, res) {
  console.log(res)
})

### mapAsync(iterator, mapFn)

var mapIterator = iterators.map(someNumberIterator, function(err, each, cb) {
  cb(null, each * 2)
})

Create an iterator that filters the values of the source iterator using a filter function.

filter(iterator, filterFn)

var evenNumbersIterator = iterators.filter(someNumberIterator, function(err, each) {
  return (each % 2) == 0
})

### filterAsync(iterator, filterFn)

var evenNumbersIterator = iterators.filter(someNumberIterator, function(err, each, cb) {
  cb(null, (each % 2) == 0)
})

### range(iterator, range) Creates an iterator that only iteratores over the specified range.

range is specified as {from: startIndex, to: endIndex} where from and to are both inclusive.

var rangeIterator = iterators.range(iterator, {from: 10, to: 19})

### buffer(iterator, bufferSize) Creates an iterator with an internal buffer that is always filled until `bufferSize`. The buffer can abviously only grow if the buffer iterator is read slower than the underlying iterator source can return data.

The current buffer fill ratio can be inspected at any time using bufferFillRatio() which returns a number between 0..1.

The buffer size can be changed using setBufferSize(bufferSize).

var bufferedIterator = iterators.buffer(someIterator, 10)

// inspect buffer size
console.log(bufferedIterator.bufferFillRatio())

// change the buffer size later
bufferedIterator.setBufferSize(100)

##Iterator Targets

### toArray(iterator, cb) Reads the source iterator and writes the results to an array.

iterators.toArray(someIterator, function(err, array) {
  console.log(array)
})

### toWritableStream(iterator, writeStream, encoding, cb) Reads the source iterator and writes the result to a [Writable Stream](http://nodejs.org/api/stream.html#stream_class_stream_writable).

var writeStream = fs.createWriteStream('output.txt')
iterators.toWritableStream(iterator, writeStream, 'utf8', function() {
  console.log('done')
})

##Utilities

### forEach(iterator, fn, cb) Reads the source iterator and invokes `fn` for each value of the iterator.

iterators.forEach(someIterator, function(err, data) {
  console.log(data)
}, function() {
  console.log('end')
})

### forEachAsync(iterator, fn, cb) Reads the source iterator and invokes `fn` for each value of the iterator. Only once the callback is invoked the next value is read from the source iterator.

iterators.forEachAsync(someIterator, function(err, data, cb) {
  console.log(data)
  setTimeout(cb, 100)
}, function() {
  console.log('end')
})

##Other libraries

Some libraries using the async iterator pattern:

• stream-iterator - wrap any stream into an async iterator
• node-leveldown - allows you to iterate over entries in LevelDB

##Contributors This project was created by Mirko Kiefer (@mirkokiefer).