batch

What is batch?

The 'batch' npm package is a utility that allows for batch processing of jobs, where jobs can be executed in parallel or serially. It is useful for managing and controlling the execution of multiple tasks at once, with features such as concurrency control, error handling, and event listening.

What are batch's main functionalities?

Batch Processing

This code sample demonstrates how to create a batch of jobs with a concurrency of 2, meaning two jobs will be processed at a time. It also shows how to listen for progress events and handle the completion of all jobs.

const Batch = require('batch');
const batch = new Batch();

batch.concurrency(2);

batch.push(function(done) {
  setTimeout(function() {
    console.log('Job 1 done');
    done();
  }, 1000);
});

batch.push(function(done) {
  setTimeout(function() {
    console.log('Job 2 done');
    done();
  }, 500);
});

batch.on('progress', function(e) {
  console.log(e.percent + '% complete');
});

batch.end(function(err, results) {
  console.log('All jobs completed');
});

Error Handling

This code sample demonstrates how to handle errors within a batch. If any job calls the 'done' callback with an error, the 'end' event will receive this error, allowing for centralized error handling.

const Batch = require('batch');
const batch = new Batch();

batch.push(function(done) {
  // Simulate an error
  done(new Error('Something went wrong'));
});

batch.end(function(err, results) {
  if (err) {
    console.error('Error encountered:', err.message);
  }
  console.log('Results:', results);
});

Other packages similar to batch

async

The 'async' package provides a collection of powerful functions for working with asynchronous JavaScript. It offers similar batch processing capabilities with functions like 'parallel', 'series', and 'queue'. Compared to 'batch', 'async' offers a broader set of features for controlling the flow of asynchronous operations.

p-map

The 'p-map' package is a promise-based map function that supports concurrency control, similar to the concurrency feature in 'batch'. It is designed to run multiple promise-returning & async functions with limited concurrency. 'p-map' is more modern with its promise-based interface compared to 'batch's callback-style.

bluebird

The 'bluebird' package is a fully-featured promise library with utilities that can be used for concurrency control, similar to 'batch'. It includes methods like 'map', 'each', and 'mapSeries' that can be used for batch processing. 'bluebird' is more comprehensive in terms of promise-related features and optimizations.

README

batch

Simple async batch

Installation

$ npm install batch

API

var Batch = require('batch')
  , batch = new Batch;

ids.forEach(function(id){
  batch.push(function(done){
    User.get(id, done);
  });
});

batch.on('progress', function(e){
  
});

batch.end(function(err, users){

});

Progress events

Contain the "job" index, response value, duration information, and completion data.

{ index: 1,
  value: 'bar',
  pending: 2,
  total: 3,
  complete: 2,
  percent: 66,
  start: Thu Oct 04 2012 12:25:53 GMT-0700 (PDT),
  end: Thu Oct 04 2012 12:25:53 GMT-0700 (PDT),
  duration: 0 }

License

(The MIT License)

Copyright (c) 2011 TJ Holowaychuk <[e-mail]>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.