run-parallel-limit

What is run-parallel-limit?

The run-parallel-limit npm package allows for the execution of multiple asynchronous tasks in parallel, with a limit on the number of tasks that can run simultaneously. This is particularly useful for managing resource usage and avoiding overloading the system when performing a large number of asynchronous operations, such as file I/O, network requests, or CPU-intensive tasks.

What are run-parallel-limit's main functionalities?

Running multiple tasks in parallel with a limit

This feature allows you to run multiple asynchronous tasks in parallel, but with a specified limit on the number of tasks that can be executed at the same time. In the code sample, two tasks are defined and run with a limit of 1, meaning they will execute one after the other rather than simultaneously.

const runParallelLimit = require('run-parallel-limit');

const tasks = [
  function(callback) {
    setTimeout(function() {
      callback(null, 'one');
    }, 200);
  },
  function(callback) {
    setTimeout(function() {
      callback(null, 'two');
    }, 100);
  }
];

runParallelLimit(tasks, 1, function(err, results) {
  console.log(results); // ['one', 'two']
});

Other packages similar to run-parallel-limit

async

The 'async' package provides a wide range of functions for working with asynchronous JavaScript, including parallel execution with limits similar to run-parallel-limit. However, 'async' is more comprehensive, offering additional utilities such as series, waterfall, and eachLimit for different patterns of asynchronous control flow.

p-limit

The 'p-limit' package is a promise-based tool for limiting the concurrency of asynchronous operations. It is similar to run-parallel-limit but uses Promises instead of callbacks, making it a better fit for modern asynchronous patterns in JavaScript. It allows for easy control of concurrent executions with a simple and clean API.

bluebird

Bluebird is a fully-featured Promise library that includes utilities for controlling the concurrency of promise-based operations. Its 'Promise.map' function with the 'concurrency' option can be used to achieve similar functionality to run-parallel-limit, offering a promise-based alternative with additional features like error handling and cancellation.

README

run-parallel-limit

Run an array of functions in parallel, but limit the number of tasks executing at the same time

install

npm install run-parallel-limit

usage

parallelLimit(tasks, limit, [callback])

Run the tasks array of functions in parallel, with a maximum of limit tasks executing at the same time. If any of the functions pass an error to its callback, the main callback is immediately called with the value of the error. Once the tasks have completed, the results are passed to the final callback as an array.

Note that the tasks are not executed in batches, so there is no guarantee that the first limit tasks will complete before any others are started.

It is also possible to use an object instead of an array. Each property will be run as a function and the results will be passed to the final callback as an object instead of an array. This can be a more readable way of handling the results.

arguments

• tasks - An array or object containing functions to run. Each function is passed a callback(err, result) which it must call on completion with an error err (which can be null) and an optional result value.
• limit - The maximum number of tasks to run at any time.
• callback(err, results) - An optional callback to run once all the functions have completed. This function gets a results array (or object) containing all the result arguments passed to the task callbacks.

example

var parallelLimit = require('run-parallel-limit')

var tasks = [
  function (callback) {
    setTimeout(function () {
      callback(null, 'one')
    }, 200)
  },
  function (callback) {
    setTimeout(function () {
      callback(null, 'two')
    }, 100)
  },
  ... hundreds more tasks ...
]

parallelLimit(tasks, 5, function (err, results) {
  // optional callback
  // the results array will equal ['one', 'two', ...] even though
  // the second function had a shorter timeout.
})

The above code runs with a concurrency limit of 5, so at most 5 tasks will be running at any given time.

This module is basically equavalent to async.parallelLimit, but it's handy to just have the one function you need instead of the kitchen sink. Modularity! Especially handy if you're serving to the browser and need to reduce your javascript bundle size.

Works great in the browser with browserify!

see also

• run-auto
• run-parallel
• run-series
• run-waterfall

license

MIT. Copyright (c) Feross Aboukhadijeh.