rethinkdb-job-queue

Introduction

rethinkdb-job-queue is a persistent job or task queue backed by RethinkDB. It has been build as an alternative to using a Redis backed job queue such as Kue, Bull, or Bee-Queue.

Please Star on GitHub / NPM and Watch for updates.

Documentation

For full documentation please see the wiki

Project Status

• This rethinkdb-job-queue module is fully functional.
• There are over 1100 integration tests.
• This project is complete and needs to be taken out for a spin.
• In a few months I will bump the version up to 1.0.0 to support SemVer.
• Please provide feedback or raise issues prior to the version bump.

Quick Start

Installation

Note: You will need to install RethinkDB before you can use rethinkdb-job-queue

npm install rethinkdb-job-queue --save

Simple Example

const Queue = require('rethinkdb-job-queue')
const options = {
  db: 'JobQueue', // The name of the database in RethinkDB
  name: 'Mathematics' // The name of the table in the database
}

const q = new Queue(options)

const job = q.createJob()
job.numerator = 123
job.denominator = 456

q.process((job, next) => {
    try {
      let result = job.numerator / job.denominator
      // Do something with your result
      return next(result)
    } catch (err) {
      console.error(err)
      return next(err)
    }
})

return q.addJob(job).catch((err) => {
  console.error(err)
})

E-Mail Job Example using nodemailer

// The following is not related to rethinkdb-job-queue.
// nodemailer configuration
const nodemailer = require('nodemailer')
const transporter = nodemailer.createTransport({
  service: 'Mailgun',
  auth: {
    user: '[e-mail]',
    pass: 'your-api-key-here'
  }
})

// Setup e-mail data with unicode symbols
var mailOptions = {
  from: '"Registration" <[e-mail]>', // Sender address
  subject: 'Registration', // Subject line
  text: 'Click here to complete your registration', // Plaintext body
  html: '<b>Click here to complete your registration</b>' // HTML body
}

// rethinkdb-job-queue configuration
const Queue = require('rethinkdb-job-queue')

// Queue options have defaults and are not required
const options = {
  db: 'JobQueue', // The name of the database in RethinkDB
  name: 'RegistrationEmail', // The name of the table in the database
  host: 'localhost',
  port: 28015,
  masterInterval: 300, // Database review period in seconds
  changeFeed: true, // Enables events from the database table
  concurrency: 100,
  removeFinishedJobs: 30, // true, false, or number of days.
}

// This is the main queue instantiation call
const q = new Queue(options)

// Customizing the default job options for new jobs
const jobDefaults = {
  priority: 'normal',
  timeout: 300,
  retryMax: 3, // Four attempts, first then three retries
  retryDelay: 600 // Time in seconds to delay retries
}
q.jobOptions = jobDefaults

const job = q.createJob()
// The createJob method will only create the job locally.
// It will need to be added to the queue.
// You can decorate the job with any data to be saved for processing
job.recipient = '[e-mail]'

q.process((job, next) => {
  // Send email using job.recipient as the destination address
  mailOptions.to = job.recipient
  return transporter.sendMail(mailOptions).then((info) => {
    console.dir(info)
    return next(info)
  }).catch((err) => {
    // This catch if for nodemailer sendMail errors.
    return next(err)
  })
})

return q.addJob(job).then((savedJobs) => {
  // savedJobs is an array of the jobs added with updated properties
}).catch((err) => {
  console.error(err)
})

Contributing

1. Fork it!
2. Create your feature branch: git checkout -b my-new-feature
3. Commit your changes: git commit -am 'Add some feature'
4. Push to the branch: git push origin my-new-feature
5. Submit a pull request :D

Credits

Thanks to the following marvelous packages and people for their hard work:

• The RethinkDB team for the great database.
• The RethinkDB driver rethinkdbdash by Michel
• The Promise library Bluebird by Petka Antonov.
• The date management library moment.
• The uuid package.

This list could go on...

License

MIT