atomic-queue

atomic-queue

a crash friendly queue that persists queue state and can restart. uses a worker pool and has configurable concurrency

API

for example usage see test.js

var queue = require('atomic-queue')(worker, opts)

initialize a new queue with a worker function and optional options. queue is a stream

you queue things by writing them to the queue stream:

queue.write('hello')
queue.write('goodbye')
queue.write({name: 'bob'})

worker must be a function that has this API:

function work (data, done) {
  // do work, then call done with (err) if there was an error
}

data in the worker function will be the data you wrote into the queue above

events

in addition to standard stream events you can also listen to the following:

queue.on('ready')

emitted after startup when the queue state has been read from disk and the queue is now ready to start working

queue.on('error')

when a catastrophic error has occurred

queue.on('update-start')

when the queue starts flushing its state to disk

queue.on('update-end')

when the queue finishes flushing its state to disk

queue.pool.on('start')

when a job starts working

queue.pool.on('finish')

when a job finishes working