The qjobs npm package is a lightweight, fast job queue manager for Node.js. It allows for efficient management of tasks with the ability to limit the number of concurrent jobs, making it suitable for scenarios where resource management and job scheduling are critical. It's particularly useful for managing CPU-intensive tasks or IO-bound operations in a controlled manner.

What are qjobs's main functionalities?

Job scheduling and concurrency control

This feature allows you to schedule jobs and control the number of jobs running concurrently. In the code sample, a new QJobs instance is created with a maximum concurrency of 5, meaning no more than 5 jobs will run at the same time. Jobs are added to the queue and executed with controlled concurrency.

const QJobs = require('qjobs');

const myQueue = new QJobs({maxConcurrency: 5});

const myJob = function(args, next) {
    console.log('Working on job with args', args);
    next();
};

for (let i = 0; i < 10; i++) {
    myQueue.add(myJob, [i]);
}

myQueue.run();

Job completion and error handling

This feature demonstrates how to handle job completion and errors. Jobs are defined with a condition to either complete successfully or fail. The QJobs instance listens for 'end' and 'error' events to handle job completion and errors respectively. This allows for robust error handling and notification upon job queue completion.

const QJobs = require('qjobs');

const myQueue = new QJobs();

const myJob = function(args, next) {
    if (args.shouldFail) {
        throw new Error('Job failed');
    } else {
        console.log('Job completed successfully');
        next();
    }
};

myQueue.on('end', function() {
    console.log('All jobs have been processed');
});

myQueue.on('error', function(err) {
    console.error('Error processing job:', err);
});

myQueue.add(myJob, [{shouldFail: false}]);
myQueue.add(myJob, [{shouldFail: true}]);

myQueue.run();

Other packages similar to qjobs

Readme

Source

qjobs

Efficient queue job manager module for nodejs.

Features

Concurrency limiter
Dynamic queue, a job can be added while the queue is running
Optional delay before continuing after max concurrency has been reached
Support of pause/unpause
Events emitter based: start, end, sleep, continu, jobStart, jobEnd
Quick statistic function, so you can know where the queue is, at regular interval

For what it can be usefull ?

Jobs which needs to run in parallels, but in a controled maner, example:

Network scanners
Parallels monitoring jobs
Images/Videos related jobs

Compatibility :

not tested with nodejs < 0.10

Examples

(take a look at tests directory if you are looking for running samples)

var qjobs = new require('./qjobs');
                                
// My non blocking main job     
var myjob = function(args,next) {
    setTimeout(function() {
        console.log('Do something interesting here',args);
        next();
    },1000);
}

var q = new qjobs({maxConcurrency:10});

// Let's add 30 job to the queue
for (var i = 0; i<30; i++) {
    q.add(myjob,[i,'test '+i]);
}

q.on('start',function() {
    console.log('Starting ...');
});

q.on('end',function() {
    console.log('... All jobs done');
});

q.on('jobStart',function(args) {
    console.log('jobStart',args);
});

q.on('jobEnd',function(args) {

    console.log('jobend',args);

    // If i'm jobId 10, then make a pause of 5 sec

    if (args._jobId == 10) {
        q.pause(true);
        setTimeout(function() {
            q.pause(false);
        },5000);
    }
});

q.on('pause',function(since) {
    console.log('in pause since '+since+' milliseconds');
});

q.on('unpause',function() {
    console.log('pause end, continu ..');
});

q.run();

//q.abort() will empty jobs list

Keywords

FAQs

What is qjobs?

Is qjobs popular?

Is qjobs well maintained?

Last updated on 19 Feb 2018

Did you know?

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install