batch

What is batch?

The 'batch' npm package is a utility that allows for batch processing of jobs, where jobs can be executed in parallel or serially. It is useful for managing and controlling the execution of multiple tasks at once, with features such as concurrency control, error handling, and event listening.

What are batch's main functionalities?

Batch Processing

This code sample demonstrates how to create a batch of jobs with a concurrency of 2, meaning two jobs will be processed at a time. It also shows how to listen for progress events and handle the completion of all jobs.

const Batch = require('batch');
const batch = new Batch();

batch.concurrency(2);

batch.push(function(done) {
  setTimeout(function() {
    console.log('Job 1 done');
    done();
  }, 1000);
});

batch.push(function(done) {
  setTimeout(function() {
    console.log('Job 2 done');
    done();
  }, 500);
});

batch.on('progress', function(e) {
  console.log(e.percent + '% complete');
});

batch.end(function(err, results) {
  console.log('All jobs completed');
});

Error Handling

This code sample demonstrates how to handle errors within a batch. If any job calls the 'done' callback with an error, the 'end' event will receive this error, allowing for centralized error handling.

const Batch = require('batch');
const batch = new Batch();

batch.push(function(done) {
  // Simulate an error
  done(new Error('Something went wrong'));
});

batch.end(function(err, results) {
  if (err) {
    console.error('Error encountered:', err.message);
  }
  console.log('Results:', results);
});

Other packages similar to batch

async

The 'async' package provides a collection of powerful functions for working with asynchronous JavaScript. It offers similar batch processing capabilities with functions like 'parallel', 'series', and 'queue'. Compared to 'batch', 'async' offers a broader set of features for controlling the flow of asynchronous operations.

p-map

The 'p-map' package is a promise-based map function that supports concurrency control, similar to the concurrency feature in 'batch'. It is designed to run multiple promise-returning & async functions with limited concurrency. 'p-map' is more modern with its promise-based interface compared to 'batch's callback-style.

bluebird

The 'bluebird' package is a fully-featured promise library with utilities that can be used for concurrency control, similar to 'batch'. It includes methods like 'map', 'each', and 'mapSeries' that can be used for batch processing. 'bluebird' is more comprehensive in terms of promise-related features and optimizations.

README

batch

Simple async batch with concurrency control and progress reporting.

Installation

$ npm install batch

API

var Batch = require('batch')
  , batch = new Batch;

batch.concurrency(4);

ids.forEach(function(id){
  batch.push(function(done){
    User.get(id, done);
  });
});

batch.on('progress', function(e){

});

batch.end(function(err, users){

});

Progress events

Contain the "job" index, response value, duration information, and completion data.

{ index: 1,
  value: 'bar',
  pending: 2,
  total: 3,
  complete: 2,
  percent: 66,
  start: Thu Oct 04 2012 12:25:53 GMT-0700 (PDT),
  end: Thu Oct 04 2012 12:25:53 GMT-0700 (PDT),
  duration: 0 }

License

MIT