Node Redis key scanner
A simple ES6 Redis key scanner for Node 10 and newer. This is a small class that allows you to do one thing quickly and easily: scan a Redis key space for keys that match a given pattern.
See the Redis SCAN command documentation for information about how to write patterns for matching, the guarantees, caveats, etc.
Install
With NPM:
npm install node-redis-scan
Or with Yarn:
yarn add node-redis-scan
Use
Instantiate this class with a Node Redis client (version 3.x) then perform key space scans!
⚠️ Note: Node Redis clients of version 4.x or newer are not supported by this library. At least not yet. Luckily, if you are using version 4.x you may not need this library. You could likely use a Node Redis 4.x scan iterator instead of this library.
Redis supports scanning the entire key space or scanning hashes, sets, and sorted sets with the HSCAN
, SSCAN
, and ZSCAN
commands, respectively. All of this functionality is available by calling the appropriately named functions listed below.
Table of contents
The scan()
method
The scan()
method provides the easiest way to scan your key space with a single callback that will be passed all matching keys. Depending on the size of your key space (millions of keys and beyond) this process might take many seconds or longer.
Parameters
Name | Type | Description |
---|
pattern | string | The Redis glob-style string pattern to match keys against. |
options | object (optional) | An object for configuring the precise scan parameters. Available options:
method - String name for which underlying Redis scan method we want to use. Defaults to 'scan' and can be set to one of 'hscan', 'sscan', or 'zscan'.key - The string name of the applicable key. Required if the method is set to 'hscan', 'sscan', or 'zscan'.count - A number representing how much work Redis should do with each iteration of the given scan command. This is useful if you want to scan a huge key space faster. The trade off is lengthening the brief segments of time that Redis is locked doing work scanning. See the Redis COUNT option documentation.type - A string name of a Redis key type. This is used for searching for keys of a certain type. See the Redis TYPE option documentation.limit - A number representing a limit for how many results should be returned. Because of the nature of the Redis SCAN command plus the interaction with the count option we can never guarantee returning this exact limit. When the limit is reached or exceeded the scan halts and is considered complete.
|
callback | function | Invoked with (err, matchingKeys). |
Example
const redis = require('redis');
const redisScan = require('node-redis-scan');
const client = redis.createClient();
const scanner = new redisScan(client);
scanner.scan('some-pattern*', (err, matchingKeys) => {
if (err) throw(err);
console.log(matchingKeys);
});
scanner.scan('*another-pattern', {count: 1000}, (err, matchingKeys) => {
if (err) throw(err);
console.log(matchingKeys);
});
The eachScan()
method
The eachScan()
method is useful if you want to perform work with matched keys at the same time as the key space is being scanned. When you’re scanning an enormous key space this is likely a more efficient way to operate: you can begin handling matched keys asynchronously, before the entire scan has finished. Unfortunately this approach doesn’t help in situations where you need to have every matching key prior to performing the next step in your operation/application.
Matching keys are passed to the intermediate callback function after each iteration of the Redis SCAN
command. The final callback is passed a count of how many matching keys were returned.
Parameters
Name | Type | Description |
---|
pattern | string | The Redis glob-style string pattern to match keys against. |
options | object (optional) | An object for configuring the precise scan parameters. Available options:
method - String name for which underlying Redis scan method we want to use. Defaults to 'scan' and can be set to one of 'hscan', 'sscan', or 'zscan'.key - The string name of the applicable key. Required if the method is set to 'hscan', 'sscan', or 'zscan'.count - A number representing how much work Redis should do with each iteration of the given scan command. This is useful if you want to scan a huge key space faster. The trade off is lengthening the brief segments of time that Redis is locked doing work scanning. See the Redis COUNT option documentation.type - A string name of a Redis key type. This is used for searching for keys of a certain type. See the Redis TYPE option documentation.limit - A number representing a limit for how many results should be returned. Because of the nature of the Redis SCAN command plus the interaction with the count option we can never guarantee returning this exact limit. When the limit is reached or exceeded the scan halts and is considered complete.
|
eachScanCallback | function | This intermediate callback is used for handling matching keys as they are returned. This function can also signal cancellation of the overall scan, if desired, by returning boolean true .
Invoked with (matchingKeys). |
callback | function | Invoked with (err, matchCount). |
Example
const redis = require('redis');
const redisScan = require('node-redis-scan');
const client = redis.createClient();
const scanner = new redisScan(client);
scanner.eachScan('some-pattern*', (matchingKeys) => {
if (matchingKeys.length) {
console.log(matchingKeys);
}
}, (err, matchCount) => {
if (err) throw(err);
console.log(`Found ${matchCount} keys.`);
});
The hscan()
and eachHScan()
methods
Using hscan()
will return all matching keys along with their values from the given hash. Note that the nature of HSCAN
is to return the keys and their values.
Example
scanner.hscan('name-of-hash', 'some-pattern*', (err, matchingKeysValues) => {
if (err) return done(err);
console.log(matchingKeysValues);
});
scanner.eachHScan('name-of-hash', 'some-pattern*', (matchingKeysValues) => {
if (matchingKeysValues.length) {
console.log(matchingKeysValues);
}
}, (err, matchCount) => {
if (err) throw(err);
console.log(`Found ${matchCount} keys and values.`);
});
The sscan()
and eachSScan()
methods
Using sscan()
will return all matching members from the given set.
Example
scanner.sscan('name-of-set', 'some-pattern*', (err, matches) => {
if (err) return done(err);
console.log(matches);
});
The zscan()
and eachZScan()
methods
Using zscan()
will return all matching members along with their scores from a given sorted set. Note that the nature of ZSCAN
is to return the members and their scores.
Example
scanner.zscan('name-of-sorted-set', 'some-pattern*', (err, matchingMembersScores) => {
if (err) return done(err);
console.log(matchingMembersScores);
});
Canceling a scan before it has finished
Using the limit
option with either the scan()
or eachScan()
method allows us to halt or cancel a scan after a certain "limit" of matching keys have been found. Note that we might receive more matching keys than the specified limit
because of the nature of the Redis SCAN command.
Example
scanner.scan('some-pattern*', {limit: 5}, (err, matchingKeys) => {
if (err) throw(err);
console.log(matchingKeys);
});
Alternatively, we can use the eachScanCallback
function parameter of the eachScan()
method for more fine-grained cancellation of a scan.
Example
scanner.eachScan(scanPattern, (matchingKeys) => {
if (Math.random() > 0.5) {
return true;
}
}, (err, matchCount) => {
if (err) throw(err);
console.log(`Found ${matchCount} keys after canceling the scan arbitrarily.`);
});
Test
Tests are run via Istanbul and Mocha. Clone the project then run:
npm run test
Contribute
Simply open an issue or send a pull request. Not sure how to do that? Check out Github’s fast and free course on how to contribute to a project.
License
Licensed under the Apache License 2.0.