bokeh

Bokeh

bokeh (pronounced boh-kay) is a simple, scalable and blazing-fast task queue built on Node.js and ZeroMQ. It allows you to offload tasks from your main application process and distribute them among a pool of workers. Workers can be running on the same host as your application, or scaled out onto multiple machines for greater processing power.

When you want a worker to run a task, just submit it to the broker using the client API. A task is simply any class in your application which responds to the run method.

Bokeh consists of three components:

• The client library which your application uses to submit tasks to the broker.
• The broker process which manages the pool of workers.
• The worker processes which are responsible for running tasks.

Installation

ZeroMQ

The only prerequisite is that you have ZeroMQ installed.

OS X

Install ZeroMQ using brew:

$ brew install zeromq

Ubuntu 10.04 LTS

Install ZeroMQ using Chris Lea's PPA:

$ sudo add-apt-repository ppa:chris-lea/zeromq
$ sudo apt-get update
$ sudo apt-get install libzmq-dbg libzmq-dev libzmq1

Bokeh

Install Bokeh using npm:

$ npm install bokeh

Overview

Task

A task is a class which responds to the run method. A task is dealt to a worker and executed.

Once the task has been completed, you must call the callback with any data you want to pass back to your application.

class Reverse
  run: (data, callback) -> callback null, data.split("").reverse().join("")

Client

The client is used by your application to submit tasks to the broker and monitor their progress.

bokeh = require "bokeh"
handle = bokeh.getClient().submitTask "Reverse", "hello world"
handle.on "complete", (data) -> console.log "Task %s completed: %s", handle.id, data
handle.on "error", (error) -> console.error "Task %s failed: %s", handle.id, error

Broker

The broker is responsible for routing messages from clients, persisting them to the data store and dealing them to workers.

bokeh = require "bokeh"
broker = new bokeh.Broker

Bokeh supports pluggable data stores for persisting tasks, currently in-memory, Redis and Riak are supported.

Worker

A worker is a process which receives tasks from a broker and executes them. You must register all your task classes with the worker.

bokeh = require "bokeh"
worker = new bokeh.Worker
worker.registerTask "Reverse", require("./tasks/reverse")

License

Bokeh is released under the MIT license.

Changelog

1.0.3 (2019-02-19)

• Fix issue loading stores on linux