redis-smq

[!IMPORTANT] Currently, RedisSMQ is going under heavy development. Pre-releases at any time may introduce new commits with breaking changes. The master branch always reflects the most recent changes. To view the latest release reference see RedisSMQ v7.2.3

RedisSMQ

RedisSMQ is a Node.js library for queuing messages (aka jobs) and processing them asynchronously with consumers. Backed by Redis, it allows scaling up your typical applications with ease of use.

Features

• High-performance message processing.
• Flexible Producer/Consumer model which offers Multi-Queue Producers & Multi-Queue Consumers.
• RedisSMQ offers different exchange types: Direct Exchange, Topic Exchange, and FanOut Exchange for publishing a message to one or multiple queues.
• Supports Point-2-Point and Pub/Sub delivery models.
• Both delivery models are reliable. For cases of failure, while delivering/consuming messages, at-least-once and at-most-once modes may be configured.
• 3 queuing strategies: FIFO queues, LIFO queues, and Priority Queues.
• Message Handler Worker Threads which allow sandboxing and running your message handler from a separate isolated thread and without affecting the performance of other message handlers from the same consumer.
• Messages can be set to expire when not delivered within a given amount of time or to have a consumption timeout while being in process.
• Queues may be rate Limited to control the rate at which the messages are consumed.
• Has a builtin scheduler allowing messages to be delayed, to be delivered for N times with an optional period between deliveries, or simply to be scheduled using CRON expressions.
• Provides an HTTP API to interact with the message queue for external services.
• RedisSMQ can be managed also from your web browser.
• Either node-redis or ioredis can be used as a Redis client.
• Highly optimized, implemented using pure callbacks, with small memory footprint and no memory leaks. See Callback vs Promise vs Async/Await benchmarks.
• Both ESM & CJS modules are supported.

RedisSMQ Use Case: Multi-Queue Producers & Multi-Queue Consumers

What's new?

:rocket: RedisSMQ v8 is coming soon!

Installation

npm i redis-smq@rc

Considerations:

• Minimal Node.js version is >= 18 (RedisSMQ is tested under current active LTS and maintenance LTS Node.js releases).
• Minimal Redis server version is 4.0.0.

Usage

RedisSMQ provides 3 classes in order to work with the message queue: ProducibleMessage, Producer, and Consumer.

Producers and consumers exchange data using one or multiple queues that may be created using the Queue Class.

A queue is responsible for holding messages which are produced by producers and are delivered to consumers.

Creating a queue

const queue = new Queue();
queue.save(
  'my_queue',
  EQueueType.LIFO_QUEUE,
  EQueueDeliveryModel.POINT_TO_POINT,
  (err) => {
    if (err) console.error(err);
  },
);

In the example above we are defining a LIFO queue with a POINT-2-POINT delivery model.

See Queues for more details.

Producing a message

const msg = new ProducibleMessage();
msg.setQueue('my_queue').setBody('Hello Word!');
producer.produce(msg, (err, ids) => {
  if (err) console.error(err);
  else console.log(`Produced message IDs are: ${ids.join(', ')}`);
});

See Producing Messages for more details.

Consuming a message

const consumer = new Consumer();
const messageHandler = (msg, cb) => {
  console.log(msg.body);
  cb();
};
consumer.consume('my_queue', messageHandler, (err) => {
  if (err) console.error(err);
});

See Consuming Messages for more details.

Documentation

See RedisSMQ Docs for more details.

Contributing

So you are interested in contributing to this project? Please see CONTRIBUTING.md.

License

MIT