multiple-redis

multiple-redis

Run redis commands against multiple redis instances.

• Overview
  • Why?
  • How This Library Works
  • Simple Scenario
• Usage
• Debug
• Installation
• Limitations
• API Documentation
• Contributing
• Release History
• License

Overview

This library enables to submit redis commands to multiple redis instances.
The client interface is the same as the 'redis' node package at: node redis
However, every command actually invokes multiple redis backends.

Why

Generally in production you would like a failover capability for your redis server.
Working with only one redis instance can cause your entire production system to fail in case redis goes down for any reason.
For example, when using redis as an express session store and redis is down, users HTTP requests will be rejected.
So how can we setup high availability?
You could setup redis master and redis slave. That would have those two redis servers sync data between them and you can read it from both.
But what if the master redis goes down? If you didn't setup the slave as writable, you are in a possible problem until the master is up.
If you did, again there is an issue when master goes back up.
Not to mention you have to setup a deployment installation process to set one redis for master and one for slave (well... at least one). So one solution is to be able to change the master in runtime in such a scenario, so now you have a choice to deploy sentinal.
But that is even more work on the production team and what if sentinal goes down? Who monitors the monitor?
So another choice is to let the app code detect that the master is down and let it send the commands to select a new master from the existing slaves.
But you have multiple instances of your app, and they can all try to select a new master and cause issues for your redis servers.
So existing solutions don't work? Of course they do. But they are a pain to configure, manage and monitor in production.
This is where this library comes in.
What if you had multiple standalone redis servers that didn't know of each other?
Your app ensures they are sync or at least it knows to pick up the data from the correct place.
If one goes down, no problem. You are running with the redis servers that are still up and they too hold the data you need.
Important to explain, this does not mean that I do not support the Redis master/slave + sentinal solution but sometimes using multiple independent Redis servers serves as a better solution, which is much more simple to deploy and manage in production.

How This Library Works

This library basically does 2 main things.

• Get commands - When a get (which does not modify data) command is called, the redis client will go redis by redis in a sequence until it finds a redis which provides data for the get command.
  Any error, or any redis which is unable to provide the data is ignored.
  Once a specific redis provides the data, the next redis servers are skipped and the command callback is invoked with that data.
• Set/Other commands - When a non 'get' command is called, all redis servers are invoked in parallel with the same command to ensure that all redis servers are updated with latest data.
  If at least redis server was able to process the command, the original command callback will receive a valid response.
  This means that at least once redis server needs to work good for the main client to notify the calling code that everything works ok.
  To ensure you don't get outdated data, the redis servers should work without persistence to disk.
  So when a redis server goes back up, it is empty. But you still get the data from the other redis that was up and holds that data.
  When that key gets updated with new data, both redis servers are now holding the latest version.
  A side affect of this solution is that every publish event would be received multiple times by the subscribers, so you need to code accordingly and prevent any possible issues from handling duplicate published messages.

Simple Scenario

Let us take the express redis session store as an example.
Since this library provides the same redis interface as the common redis client, you can provide this library to the redis session store.
When a new session is created, it will be created in all redis servers (in this example, lets assume we have 2).
In case the first redis server suddenly fails, the session store is still able to fetch and update the session data from the second redis server.
When the first redis server comes back up, the session is still available to the session store from the second redis server and any session modification (due to some HTTP request) will cause both redis servers to now hold the latest express session data.
It is by no means, a perfect solution, but it does has its advantages.
First and foremost, its simple deployment requirements.

Usage

In order to use this library, you need to either create the redis clients or provide the redis connection data as follows:

//create multiple redis clients
var redis = require('redis');
var client1 = redis.createClient(...);
var client2 = redis.createClient(...);

//create the wrapper client
var MultipleRedis = require('multiple-redis');
var multiClient = MultipleRedis.createClient([client1, client2]);

multiClient.once('all-ready', function onReady() {
  //run any command on the multi client instead of the original clients
  multiClient.set('string key', 'string val', callback);
});

Or

//create the wrapper client with connection info
var MultipleRedis = require('multiple-redis');
var multiClient = MultipleRedis.createClient([{
    host: 'host1',
    port: 6379
}, {
   host: 'host2',
   port: 6379
}], options);

multiClient.once('all-ready', function onReady() {
  //run any command on the multi client instead of the original clients
  multiClient.set('string key', 'string val', callback);
});

The rest of the API is the same as defined in the redis node library: node redis

Debug

In order to turn on debug messages, use the standard nodejs NODE_DEBUG environment variable.

NODE_DEBUG=multiple-redis

Installation

In order to use this library, just run the following npm install command:

npm install --save multiple-redis

Limitations

Not all of the original redis client attributes are available (for example: redis.debug_mode).
Also Multi is currently not supported.

API Documentation

See full docs at: API Docs

Contributing

See contributing guide

Release History

Date	Version	Description
2020-05-12	v2.0.0	Migrate to github actions and upgrade minimal node version
2020-02-10	v1.3.0	Upgrade to redis 3
2019-06-24	v1.2.1	Added 'forceParallel' option to issue GET in parallel #5
2017-07-26	v1.0.61	Added all-ready event and allConnected attribute
2017-01-18	v1.0.29	setnx type commands now run in sequence and not in parallel
2015-10-22	v0.0.16	Timeout child commands (see childCommandTimeout option)
2015-09-23	v0.0.7	Upgrade to redis 2.0
2015-09-03	v0.0.3	Added support for connected and server_info attributes.
2015-09-03	v0.0.2	Initial release.

License

Developed by Sagie Gur-Ari and licensed under the Apache 2 open source license.