promise-queue

What is promise-queue?

The promise-queue npm package is designed to manage the execution of Promises in a queue with a configurable concurrency limit. This means that it allows for the sequential or controlled concurrent execution of asynchronous operations, which is particularly useful in scenarios where executing too many operations at once could lead to resource exhaustion or when the order of operations matters.

What are promise-queue's main functionalities?

Creating a queue and adding tasks

This code demonstrates how to create a new queue with a maximum of 2 concurrent operations and an infinite queue size. It then adds an asynchronous function to the queue and logs the result when the promise resolves.

const Queue = require('promise-queue');
let maxConcurrent = 2;
let maxQueue = Infinity;
let queue = new Queue(maxConcurrent, maxQueue);

function someAsyncFunction() {
  return new Promise((resolve) => {
    setTimeout(() => {
      resolve('done');
    }, 1000);
  });
}

queue.add(someAsyncFunction).then(console.log);

Handling results and errors

This snippet shows how to handle both successful results and errors for tasks added to the queue. It uses `.then` for success callbacks and `.catch` for error handling.

queue.add(someAsyncFunction).then(result => {
  console.log('Result:', result);
}).catch(error => {
  console.error('Error:', error);
});

Getting the size of the queue

These lines of code demonstrate how to get the number of queued and pending promises in the queue. `getQueueLength` returns the number of queued (not yet started) tasks, while `getPendingLength` returns the number of tasks currently running.

console.log(queue.getQueueLength());
console.log(queue.getPendingLength());

Other packages similar to promise-queue

p-queue

p-queue is a similar package that also manages promise execution with concurrency control. Compared to promise-queue, p-queue offers more advanced features such as priority levels for tasks, pausing and resuming the queue, and better overall flexibility in managing tasks.

async

The async package provides a wide range of functions for working with asynchronous JavaScript, including queue management. While not exclusively focused on promises (it supports both callbacks and promises), it offers a similar queue functionality with concurrency control. It's more versatile but can be more complex to use for promise-specific scenarios.

bottleneck

Bottleneck is another npm package designed to rate limit the execution of functions. It's similar to promise-queue in that it can help manage the execution rate of asynchronous operations, but it focuses more on limiting the execution rate to avoid hitting rate limits or overloading resources, rather than managing a queue of tasks.

README

promise-queue

Promise-based queue

Installation

promise-queue can be installed using npm:

npm install promise-queue

Interface

• new Queue(Number maxConcurrent, Number maxQueued): Queue
• Queue#add(Function generator): Promise - adds function argument that generates a promise to the queue
• Queue#getQueueLength(): Number - returns current length of buffer(added but not started promise generators) it <= maxQueued
• Queue#getPendingLength(): Number - returns number of pending(concurrently running) promises it <= maxConcurrent

Example

Configure queue

By default Queue tries to use global Promises, but you can specify your own promises.

Queue.configure(require('vow').Promise);

Or use old-style promises approach:

Queue.configure(function (handler) {
    var dfd = $.Deferred();
    try {
        handler(dfd.resolve, dfd.reject, dfd.notify);
    } catch (e) {
        dfd.reject(e);
    }
    return dfd.promise();
});

Queue one by one example

var maxConcurrent = 1;
var maxQueue = Infinity;
var queue = new Queue(maxConcurrent, maxQueue);

app.get('/version/:user/:repo', function (req, res, next) {
    queue.add(function () {
        // Assume that this action is a way too expensive
        // Call of this function will be delayed on second request
        return downloadTarballFromGithub(req.params);
    })
    .then(parseJson('package.json'))
    .then(function (package) {
        res.send(package.version);
    })
    .catch(next);
});

Getting number of pending promises and queue(buffered promises) length

var maxConcurrent = 1;
var maxQueue = 1;
var queue = new Queue(maxConcurrent, maxQueue);

queue.add(function () {
    queue.getQueueLength() === 0;
    queue.getPendingLength() === 1;
    return somePromise();
});

queue.add(function () {
    queue.getQueueLength() === 0;
    queue.getPendingLength() === 0;
    return somePromise();
});

queue.getQueueLength() === 1;
queue.getPendingLength() === 1;

Live example