
orkid

Reliable and modern Redis-Streams based task queue for Node.js.
Screenshot

Table of Contents
Features
Requirements
👏 Important: Redis-Streams feature is not available before Redis version 5.
Install
npm install orkid --save
Examples
Basic example of producing and consuming tasks:
Producing tasks:
const { Producer } = require('orkid');
const producer = new Producer('basic');
async function addTasks() {
for (let i = 0; i < 10; i++) {
console.log('Adding task ', i);
await producer.addTask(i);
}
}
addTasks()
.then(() => {
console.log('Done');
process.exit();
})
.catch(e => console.error(e));
Consuming tasks:
const { Consumer } = require('orkid');
async function workerFn(data, metadata) {
let result;
console.log('Task done!');
return result;
}
const consumer = new Consumer('basic', workerFn);
consumer.start();
👏 More examples are available in the ./examples directory, including how to do task de-duplication, retry on failure, timeout etc. 👏
API Documentation
API Documentation is available here.
Monitoring and Management UI/Admin Panel

You need to run orkid-ui
separately for the dashboard. Detail instructions on how to run orkid-ui
locally or in production using docker/docker-compose can be found here:
https://github.com/mugli/orkid-ui#running-locally
Task/job life-cycle
[TODO: Add a flowchart here]
FAQ
Is this production ready?
This project is under active development right now. API WILL introduce breaking changes until we reach version 1.0. After that semantic versioning will be followed.
Why a new job queue for Node.js?
- All the redis-based solutions were created before Redis-Streams (https://redis.io/topics/streams-intro) became available. They all require a lot of work on the queue-side to ensure durability and atomicity of jobs handling operations. Redis-Streams was specifically designed to made this kind of tasks easier, thus allows simpler core in the queue and more reliable and maintainable operations.
- None of existing usable job queues in Node.js offers a monitoring option that we liked.
- None of the existing usable task queues support task de-duplication.
How do I set priority in the tasks?
Redis-Streams isn't a right primitive to make a priority queue efficiently on top of it. Orkid doesn't support priority queues now and probably never will.
However, as a workaround, you can create a separate queue, keep its workload minimal and use it for high priority jobs with Orkid.
What is the order of job/task delivery?
Jobs are processed in the order they are produced. However, if retry option is turned on and is applicable, failed jobs gets enqueued to the queue at once, along with other newly produced jobs.
Maintainer(s)
License
MIT
Related Projects
- orkid-ui: Dashboard to monitor and manage Orkid task queue.
- orkid-api: GraphQL API to monitor and manage Orkid task queue (used internally by orkid-ui).