redis-parser
Advanced tools
Comparing version 2.6.0 to 3.0.0
@@ -0,1 +1,19 @@ | ||
# Changelog | ||
## v.3.0.0 - 25 May, 2017 | ||
Breaking Changes | ||
- Drop support for Node.js < 4 | ||
- Removed support for hiredis completely | ||
Internals | ||
- Due to the changes to ES6 the error performance improved by factor 2-3x | ||
- Improved length calculation performance (bulk strings + arrays) | ||
Features | ||
- The parser now handles weird input graceful | ||
## v.2.6.0 - 03 Apr, 2017 | ||
@@ -5,3 +23,3 @@ | ||
- Use Buffer.allocUnsafe instead of new Buffer() with modern Node.js versions | ||
- Use Buffer.allocUnsafe instead of new Buffer() with modern Node.js versions | ||
@@ -12,8 +30,8 @@ ## v.2.5.0 - 11 Mar, 2017 | ||
- Added a `ParserError` class to differentiate them to ReplyErrors. The class is also exported | ||
- Added a `ParserError` class to differentiate them to ReplyErrors. The class is also exported | ||
Bugfixes | ||
- All errors now show their error message again next to the error name in the stack trace | ||
- ParserErrors now show the offset and buffer attributes while being logged | ||
- All errors now show their error message again next to the error name in the stack trace | ||
- ParserErrors now show the offset and buffer attributes while being logged | ||
@@ -24,3 +42,3 @@ ## v.2.4.1 - 05 Feb, 2017 | ||
- Fixed minimal memory consumption overhead for chunked buffers | ||
- Fixed minimal memory consumption overhead for chunked buffers | ||
@@ -31,11 +49,11 @@ ## v.2.4.0 - 25 Jan, 2017 | ||
- Added `reset` function to reset the parser to it's initial values | ||
- Added `setReturnBuffers` function to reset the returnBuffers option (Only for the JSParser) | ||
- Added `setStringNumbers` function to reset the stringNumbers option (Only for the JSParser) | ||
- All Errors are now of sub classes of the new `RedisError` class. It is also exported. | ||
- Improved bulk string chunked data handling performance | ||
- Added `reset` function to reset the parser to it's initial values | ||
- Added `setReturnBuffers` function to reset the returnBuffers option (Only for the JSParser) | ||
- Added `setStringNumbers` function to reset the stringNumbers option (Only for the JSParser) | ||
- All Errors are now of sub classes of the new `RedisError` class. It is also exported. | ||
- Improved bulk string chunked data handling performance | ||
Bugfixes | ||
- Parsing time for big nested arrays is now linear | ||
- Parsing time for big nested arrays is now linear | ||
@@ -46,3 +64,3 @@ ## v.2.3.0 - 25 Nov, 2016 | ||
- Parsing time for big arrays (e.g. 4mb+) is now linear and works well for arbitrary array sizes | ||
- Parsing time for big arrays (e.g. 4mb+) is now linear and works well for arbitrary array sizes | ||
@@ -62,7 +80,7 @@ This case is a magnitude faster than before | ||
- Improve `stringNumbers` parsing performance by up to 100% | ||
- Improve `stringNumbers` parsing performance by up to 100% | ||
Bugfixes | ||
- Do not unref the interval anymore due to issues with NodeJS | ||
- Do not unref the interval anymore due to issues with NodeJS | ||
@@ -73,3 +91,3 @@ ## v.2.1.1 - 31 Oct, 2016 | ||
- Remove erroneously added const to support Node.js 0.10 | ||
- Remove erroneously added const to support Node.js 0.10 | ||
@@ -80,5 +98,5 @@ ## v.2.1.0 - 30 Oct, 2016 | ||
- Improve parser errors by adding more detailed information to them | ||
- Accept manipulated Object.prototypes | ||
- Unref the interval if used | ||
- Improve parser errors by adding more detailed information to them | ||
- Accept manipulated Object.prototypes | ||
- Unref the interval if used | ||
@@ -89,3 +107,3 @@ ## v.2.0.4 - 21 Jul, 2016 | ||
- Fixed multi byte characters getting corrupted | ||
- Fixed multi byte characters getting corrupted | ||
@@ -96,3 +114,3 @@ ## v.2.0.3 - 17 Jun, 2016 | ||
- Fixed parser not working with huge buffers (e.g. 300 MB) | ||
- Fixed parser not working with huge buffers (e.g. 300 MB) | ||
@@ -103,3 +121,3 @@ ## v.2.0.2 - 08 Jun, 2016 | ||
- Fixed parser with returnBuffers option returning corrupted data | ||
- Fixed parser with returnBuffers option returning corrupted data | ||
@@ -110,3 +128,3 @@ ## v.2.0.1 - 04 Jun, 2016 | ||
- Fixed multiple parsers working concurrently resulting in faulty data in some cases | ||
- Fixed multiple parsers working concurrently resulting in faulty data in some cases | ||
@@ -122,11 +140,11 @@ ## v.2.0.0 - 29 May, 2016 | ||
- Improved performance by up to 15x as fast as before | ||
- Improved options validation | ||
- Added ReplyError Class | ||
- Added parser benchmark | ||
- Switched default parser from hiredis to JS, no matter if hiredis is installed or not | ||
- Improved performance by up to 15x as fast as before | ||
- Improved options validation | ||
- Added ReplyError Class | ||
- Added parser benchmark | ||
- Switched default parser from hiredis to JS, no matter if hiredis is installed or not | ||
Removed | ||
- Deprecated hiredis support | ||
- Deprecated hiredis support | ||
@@ -137,4 +155,4 @@ ## v.1.3.0 - 27 Mar, 2016 | ||
- Added `auto` as parser name option to check what parser is available | ||
- Non existing requested parsers falls back into auto mode instead of always choosing the JS parser | ||
- Added `auto` as parser name option to check what parser is available | ||
- Non existing requested parsers falls back into auto mode instead of always choosing the JS parser | ||
@@ -145,4 +163,4 @@ ## v.1.2.0 - 27 Mar, 2016 | ||
- Added `stringNumbers` option to make sure all numbers are returned as string instead of a js number for precision | ||
- The parser is from now on going to print warnings if a parser is explicitly requested that does not exist and gracefully chooses the JS parser | ||
- Added `stringNumbers` option to make sure all numbers are returned as string instead of a js number for precision | ||
- The parser is from now on going to print warnings if a parser is explicitly requested that does not exist and gracefully chooses the JS parser | ||
@@ -153,2 +171,2 @@ ## v.1.1.0 - 26 Jan, 2016 | ||
- The parser is from now on going to reset itself on protocol errors | ||
- The parser is from now on going to reset itself on protocol errors |
'use strict' | ||
module.exports = require('./lib/parser') | ||
module.exports.ReplyError = require('./lib/replyError') | ||
module.exports.RedisError = require('./lib/redisError') | ||
module.exports.ParserError = require('./lib/redisError') |
'use strict' | ||
var StringDecoder = require('string_decoder').StringDecoder | ||
var decoder = new StringDecoder() | ||
var ReplyError = require('./replyError') | ||
var ParserError = require('./parserError') | ||
var bufferPool = bufferAlloc(32 * 1024) | ||
const Buffer = require('buffer').Buffer | ||
const StringDecoder = require('string_decoder').StringDecoder | ||
const decoder = new StringDecoder() | ||
const errors = require('redis-errors') | ||
const ReplyError = errors.ReplyError | ||
const ParserError = errors.ParserError | ||
var bufferPool = Buffer.allocUnsafe(32 * 1024) | ||
var bufferOffset = 0 | ||
@@ -12,22 +14,11 @@ var interval = null | ||
var notDecreased = 0 | ||
var isModern = typeof Buffer.allocUnsafe === 'function' | ||
/** | ||
* For backwards compatibility | ||
* @param len | ||
* @returns {Buffer} | ||
* Used for integer numbers only | ||
* @param {JavascriptRedisParser} parser | ||
* @returns {undefined|number} | ||
*/ | ||
function bufferAlloc (len) { | ||
return isModern ? Buffer.allocUnsafe(len) : new Buffer(len) | ||
} | ||
/** | ||
* Used for lengths and numbers only, faster perf on arrays / bulks | ||
* @param parser | ||
* @returns {*} | ||
*/ | ||
function parseSimpleNumbers (parser) { | ||
const length = parser.buffer.length - 1 | ||
var offset = parser.offset | ||
var length = parser.buffer.length - 1 | ||
var number = 0 | ||
@@ -42,3 +33,3 @@ var sign = 1 | ||
while (offset < length) { | ||
var c1 = parser.buffer[offset++] | ||
const c1 = parser.buffer[offset++] | ||
if (c1 === 13) { // \r\n | ||
@@ -55,11 +46,11 @@ parser.offset = offset + 1 | ||
* | ||
* The maximimum possible integer to use is: Math.floor(Number.MAX_SAFE_INTEGER / 10) | ||
* Staying in a SMI Math.floor((Math.pow(2, 32) / 10) - 1) is even more efficient though | ||
* Reading the string as parts of n SMI is more efficient than | ||
* using a string directly. | ||
* | ||
* @param parser | ||
* @returns {*} | ||
* @param {JavascriptRedisParser} parser | ||
* @returns {undefined|string} | ||
*/ | ||
function parseStringNumbers (parser) { | ||
const length = parser.buffer.length - 1 | ||
var offset = parser.offset | ||
var length = parser.buffer.length - 1 | ||
var number = 0 | ||
@@ -93,36 +84,20 @@ var res = '' | ||
/** | ||
* Returns a string or buffer of the provided offset start and | ||
* end ranges. Checks `optionReturnBuffers`. | ||
* | ||
* If returnBuffers is active, all return values are returned as buffers besides numbers and errors | ||
* | ||
* @param parser | ||
* @param start | ||
* @param end | ||
* @returns {*} | ||
*/ | ||
function convertBufferRange (parser, start, end) { | ||
parser.offset = end + 2 | ||
if (parser.optionReturnBuffers === true) { | ||
return parser.buffer.slice(start, end) | ||
} | ||
return parser.buffer.toString('utf-8', start, end) | ||
} | ||
/** | ||
* Parse a '+' redis simple string response but forward the offsets | ||
* onto convertBufferRange to generate a string. | ||
* @param parser | ||
* @returns {*} | ||
* @param {JavascriptRedisParser} parser | ||
* @returns {undefined|string|Buffer} | ||
*/ | ||
function parseSimpleString (parser) { | ||
var start = parser.offset | ||
const start = parser.offset | ||
const buffer = parser.buffer | ||
const length = buffer.length - 1 | ||
var offset = start | ||
var buffer = parser.buffer | ||
var length = buffer.length - 1 | ||
while (offset < length) { | ||
if (buffer[offset++] === 13) { // \r\n | ||
return convertBufferRange(parser, start, offset - 1) | ||
parser.offset = offset + 1 | ||
if (parser.optionReturnBuffers === true) { | ||
return parser.buffer.slice(start, offset - 1) | ||
} | ||
return parser.buffer.toString('utf8', start, offset - 1) | ||
} | ||
@@ -133,10 +108,18 @@ } | ||
/** | ||
* Returns the string length via parseSimpleNumbers | ||
* @param parser | ||
* @returns {*} | ||
* Returns the read length | ||
* @param {JavascriptRedisParser} parser | ||
* @returns {undefined|number} | ||
*/ | ||
function parseLength (parser) { | ||
var string = parseSimpleNumbers(parser) | ||
if (string !== undefined) { | ||
return string | ||
const length = parser.buffer.length - 1 | ||
var offset = parser.offset | ||
var number = 0 | ||
while (offset < length) { | ||
const c1 = parser.buffer[offset++] | ||
if (c1 === 13) { | ||
parser.offset = offset + 1 | ||
return number | ||
} | ||
number = (number * 10) + (c1 - 48) | ||
} | ||
@@ -152,7 +135,7 @@ } | ||
* | ||
* @param parser | ||
* @returns {*} | ||
* @param {JavascriptRedisParser} parser | ||
* @returns {undefined|number|string} | ||
*/ | ||
function parseInteger (parser) { | ||
if (parser.optionStringNumbers) { | ||
if (parser.optionStringNumbers === true) { | ||
return parseStringNumbers(parser) | ||
@@ -165,17 +148,16 @@ } | ||
* Parse a '$' redis bulk string response | ||
* @param parser | ||
* @returns {*} | ||
* @param {JavascriptRedisParser} parser | ||
* @returns {undefined|null|string} | ||
*/ | ||
function parseBulkString (parser) { | ||
var length = parseLength(parser) | ||
const length = parseLength(parser) | ||
if (length === undefined) { | ||
return | ||
} | ||
if (length === -1) { | ||
if (length < 0) { | ||
return null | ||
} | ||
var offsetEnd = parser.offset + length | ||
if (offsetEnd + 2 > parser.buffer.length) { | ||
parser.bigStrSize = offsetEnd + 2 | ||
parser.bigOffset = parser.offset | ||
const offset = parser.offset + length | ||
if (offset + 2 > parser.buffer.length) { | ||
parser.bigStrSize = offset + 2 | ||
parser.totalChunkSize = parser.buffer.length | ||
@@ -185,4 +167,8 @@ parser.bufferCache.push(parser.buffer) | ||
} | ||
return convertBufferRange(parser, parser.offset, offsetEnd) | ||
const start = parser.offset | ||
parser.offset = offset + 2 | ||
if (parser.optionReturnBuffers === true) { | ||
return parser.buffer.slice(start, offset) | ||
} | ||
return parser.buffer.toString('utf8', start, offset) | ||
} | ||
@@ -192,4 +178,4 @@ | ||
* Parse a '-' redis error response | ||
* @param parser | ||
* @returns {Error} | ||
* @param {JavascriptRedisParser} parser | ||
* @returns {ReplyError} | ||
*/ | ||
@@ -208,8 +194,14 @@ function parseError (parser) { | ||
* Parsing error handler, resets parser buffer | ||
* @param parser | ||
* @param error | ||
* @param {JavascriptRedisParser} parser | ||
* @param {number} type | ||
* @returns {undefined} | ||
*/ | ||
function handleError (parser, error) { | ||
function handleError (parser, type) { | ||
const err = new ParserError( | ||
'Protocol error, got ' + JSON.stringify(String.fromCharCode(type)) + ' as reply type byte', | ||
JSON.stringify(parser.buffer), | ||
parser.offset | ||
) | ||
parser.buffer = null | ||
parser.returnFatalError(error) | ||
parser.returnFatalError(err) | ||
} | ||
@@ -219,14 +211,14 @@ | ||
* Parse a '*' redis array response | ||
* @param parser | ||
* @returns {*} | ||
* @param {JavascriptRedisParser} parser | ||
* @returns {undefined|null|any[]} | ||
*/ | ||
function parseArray (parser) { | ||
var length = parseLength(parser) | ||
const length = parseLength(parser) | ||
if (length === undefined) { | ||
return | ||
} | ||
if (length === -1) { | ||
if (length < 0) { | ||
return null | ||
} | ||
var responses = new Array(length) | ||
const responses = new Array(length) | ||
return parseArrayElements(parser, responses, 0) | ||
@@ -238,9 +230,9 @@ } | ||
* | ||
* @param parser | ||
* @param elem | ||
* @param i | ||
* @param {JavascriptRedisParser} parser | ||
* @param {any[]} array | ||
* @param {number} pos | ||
* @returns {undefined} | ||
*/ | ||
function pushArrayCache (parser, elem, pos) { | ||
parser.arrayCache.push(elem) | ||
function pushArrayCache (parser, array, pos) { | ||
parser.arrayCache.push(array) | ||
parser.arrayPos.push(pos) | ||
@@ -251,11 +243,11 @@ } | ||
* Parse chunked redis array response | ||
* @param parser | ||
* @returns {*} | ||
* @param {JavascriptRedisParser} parser | ||
* @returns {undefined|any[]} | ||
*/ | ||
function parseArrayChunks (parser) { | ||
var tmp = parser.arrayCache.pop() | ||
const tmp = parser.arrayCache.pop() | ||
var pos = parser.arrayPos.pop() | ||
if (parser.arrayCache.length) { | ||
var res = parseArrayChunks(parser) | ||
if (!res) { | ||
const res = parseArrayChunks(parser) | ||
if (res === undefined) { | ||
pushArrayCache(parser, tmp, pos) | ||
@@ -271,11 +263,11 @@ return | ||
* Parse redis array response elements | ||
* @param parser | ||
* @param responses | ||
* @param i | ||
* @returns {*} | ||
* @param {JavascriptRedisParser} parser | ||
* @param {Array} responses | ||
* @param {number} i | ||
* @returns {undefined|null|any[]} | ||
*/ | ||
function parseArrayElements (parser, responses, i) { | ||
var bufferLength = parser.buffer.length | ||
const bufferLength = parser.buffer.length | ||
while (i < responses.length) { | ||
var offset = parser.offset | ||
const offset = parser.offset | ||
if (parser.offset >= bufferLength) { | ||
@@ -285,5 +277,5 @@ pushArrayCache(parser, responses, i) | ||
} | ||
var response = parseType(parser, parser.buffer[parser.offset++]) | ||
const response = parseType(parser, parser.buffer[parser.offset++]) | ||
if (response === undefined) { | ||
if (!parser.arrayCache.length) { | ||
if (!(parser.arrayCache.length || parser.bufferCache.length)) { | ||
parser.offset = offset | ||
@@ -303,4 +295,11 @@ } | ||
* Called the appropriate parser for the specified type. | ||
* @param parser | ||
* @param type | ||
* | ||
* 36: $ | ||
* 43: + | ||
* 42: * | ||
* 58: : | ||
* 45: - | ||
* | ||
* @param {JavascriptRedisParser} parser | ||
* @param {number} type | ||
* @returns {*} | ||
@@ -310,125 +309,32 @@ */ | ||
switch (type) { | ||
case 36: // $ | ||
case 36: | ||
return parseBulkString(parser) | ||
case 58: // : | ||
return parseInteger(parser) | ||
case 43: // + | ||
case 43: | ||
return parseSimpleString(parser) | ||
case 42: // * | ||
case 42: | ||
return parseArray(parser) | ||
case 45: // - | ||
case 58: | ||
return parseInteger(parser) | ||
case 45: | ||
return parseError(parser) | ||
default: | ||
return handleError(parser, new ParserError( | ||
'Protocol error, got ' + JSON.stringify(String.fromCharCode(type)) + ' as reply type byte', | ||
JSON.stringify(parser.buffer), | ||
parser.offset | ||
)) | ||
return handleError(parser, type) | ||
} | ||
} | ||
// All allowed options including their typeof value | ||
var optionTypes = { | ||
returnError: 'function', | ||
returnFatalError: 'function', | ||
returnReply: 'function', | ||
returnBuffers: 'boolean', | ||
stringNumbers: 'boolean', | ||
name: 'string' | ||
} | ||
/** | ||
* Javascript Redis Parser | ||
* @param options | ||
* @constructor | ||
*/ | ||
function JavascriptRedisParser (options) { | ||
if (!(this instanceof JavascriptRedisParser)) { | ||
return new JavascriptRedisParser(options) | ||
} | ||
if (!options || !options.returnError || !options.returnReply) { | ||
throw new TypeError('Please provide all return functions while initiating the parser') | ||
} | ||
for (var key in options) { | ||
// eslint-disable-next-line valid-typeof | ||
if (optionTypes.hasOwnProperty(key) && typeof options[key] !== optionTypes[key]) { | ||
throw new TypeError('The options argument contains the property "' + key + '" that is either unknown or of a wrong type') | ||
} | ||
} | ||
if (options.name === 'hiredis') { | ||
/* istanbul ignore next: hiredis is only supported for legacy usage */ | ||
try { | ||
var Hiredis = require('./hiredis') | ||
console.error(new TypeError('Using hiredis is discouraged. Please use the faster JS parser by removing the name option.').stack.replace('Error', 'Warning')) | ||
return new Hiredis(options) | ||
} catch (e) { | ||
console.error(new TypeError('Hiredis is not installed. Please remove the `name` option. The (faster) JS parser is used instead.').stack.replace('Error', 'Warning')) | ||
} | ||
} | ||
this.optionReturnBuffers = !!options.returnBuffers | ||
this.optionStringNumbers = !!options.stringNumbers | ||
this.returnError = options.returnError | ||
this.returnFatalError = options.returnFatalError || options.returnError | ||
this.returnReply = options.returnReply | ||
this.name = 'javascript' | ||
this.reset() | ||
} | ||
/** | ||
* Reset the parser values to the initial state | ||
* Decrease the bufferPool size over time | ||
* | ||
* Balance between increasing and decreasing the bufferPool. | ||
* Decrease the bufferPool by 10% by removing the first 10% of the current pool. | ||
* @returns {undefined} | ||
*/ | ||
JavascriptRedisParser.prototype.reset = function () { | ||
this.offset = 0 | ||
this.buffer = null | ||
this.bigStrSize = 0 | ||
this.bigOffset = 0 | ||
this.totalChunkSize = 0 | ||
this.bufferCache = [] | ||
this.arrayCache = [] | ||
this.arrayPos = [] | ||
} | ||
/** | ||
* Set the returnBuffers option | ||
* | ||
* @param returnBuffers | ||
* @returns {undefined} | ||
*/ | ||
JavascriptRedisParser.prototype.setReturnBuffers = function (returnBuffers) { | ||
if (typeof returnBuffers !== 'boolean') { | ||
throw new TypeError('The returnBuffers argument has to be a boolean') | ||
} | ||
this.optionReturnBuffers = returnBuffers | ||
} | ||
/** | ||
* Set the stringNumbers option | ||
* | ||
* @param stringNumbers | ||
* @returns {undefined} | ||
*/ | ||
JavascriptRedisParser.prototype.setStringNumbers = function (stringNumbers) { | ||
if (typeof stringNumbers !== 'boolean') { | ||
throw new TypeError('The stringNumbers argument has to be a boolean') | ||
} | ||
this.optionStringNumbers = stringNumbers | ||
} | ||
/** | ||
* Decrease the bufferPool size over time | ||
* @returns {undefined} | ||
*/ | ||
function decreaseBufferPool () { | ||
if (bufferPool.length > 50 * 1024) { | ||
// Balance between increasing and decreasing the bufferPool | ||
if (counter === 1 || notDecreased > counter * 2) { | ||
// Decrease the bufferPool by 10% by removing the first 10% of the current pool | ||
var sliceLength = Math.floor(bufferPool.length / 10) | ||
if (bufferOffset <= sliceLength) { | ||
bufferOffset = 0 | ||
} else { | ||
bufferOffset -= sliceLength | ||
} | ||
const minSliceLen = Math.floor(bufferPool.length / 10) | ||
const sliceLength = minSliceLen < bufferOffset | ||
? bufferOffset | ||
: minSliceLen | ||
bufferOffset = 0 | ||
bufferPool = bufferPool.slice(sliceLength, bufferPool.length) | ||
@@ -451,3 +357,3 @@ } else { | ||
* | ||
* @param length | ||
* @param {number} length | ||
* @returns {undefined} | ||
@@ -457,7 +363,7 @@ */ | ||
if (bufferPool.length < length + bufferOffset) { | ||
var multiplier = length > 1024 * 1024 * 75 ? 2 : 3 | ||
const multiplier = length > 1024 * 1024 * 75 ? 2 : 3 | ||
if (bufferOffset > 1024 * 1024 * 111) { | ||
bufferOffset = 1024 * 1024 * 50 | ||
} | ||
bufferPool = bufferAlloc(length * multiplier + bufferOffset) | ||
bufferPool = Buffer.allocUnsafe(length * multiplier + bufferOffset) | ||
bufferOffset = 0 | ||
@@ -478,7 +384,8 @@ counter++ | ||
* | ||
* @param parser | ||
* @param {JavascriptRedisParser} parser | ||
* @returns {String} | ||
*/ | ||
function concatBulkString (parser) { | ||
var list = parser.bufferCache | ||
const list = parser.bufferCache | ||
const oldOffset = parser.offset | ||
var chunks = list.length | ||
@@ -489,3 +396,3 @@ var offset = parser.bigStrSize - parser.totalChunkSize | ||
if (chunks === 2) { | ||
return list[0].toString('utf8', parser.bigOffset, list[0].length + offset - 2) | ||
return list[0].toString('utf8', oldOffset, list[0].length + offset - 2) | ||
} | ||
@@ -495,3 +402,3 @@ chunks-- | ||
} | ||
var res = decoder.write(list[0].slice(parser.bigOffset)) | ||
var res = decoder.write(list[0].slice(oldOffset)) | ||
for (var i = 1; i < chunks - 1; i++) { | ||
@@ -509,9 +416,10 @@ res += decoder.write(list[i]) | ||
* | ||
* @param parser | ||
* @param {JavascriptRedisParser} parser | ||
* @returns {Buffer} | ||
*/ | ||
function concatBulkBuffer (parser) { | ||
var list = parser.bufferCache | ||
const list = parser.bufferCache | ||
const oldOffset = parser.offset | ||
const length = parser.bigStrSize - oldOffset - 2 | ||
var chunks = list.length | ||
var length = parser.bigStrSize - parser.bigOffset - 2 | ||
var offset = parser.bigStrSize - parser.totalChunkSize | ||
@@ -521,3 +429,3 @@ parser.offset = offset | ||
if (chunks === 2) { | ||
return list[0].slice(parser.bigOffset, list[0].length + offset - 2) | ||
return list[0].slice(oldOffset, list[0].length + offset - 2) | ||
} | ||
@@ -528,5 +436,5 @@ chunks-- | ||
resizeBuffer(length) | ||
var start = bufferOffset | ||
list[0].copy(bufferPool, start, parser.bigOffset, list[0].length) | ||
bufferOffset += list[0].length - parser.bigOffset | ||
const start = bufferOffset | ||
list[0].copy(bufferPool, start, oldOffset, list[0].length) | ||
bufferOffset += list[0].length - oldOffset | ||
for (var i = 1; i < chunks - 1; i++) { | ||
@@ -541,67 +449,130 @@ list[i].copy(bufferPool, bufferOffset) | ||
/** | ||
* Parse the redis buffer | ||
* @param buffer | ||
* @returns {undefined} | ||
*/ | ||
JavascriptRedisParser.prototype.execute = function execute (buffer) { | ||
if (this.buffer === null) { | ||
this.buffer = buffer | ||
class JavascriptRedisParser { | ||
/** | ||
* Javascript Redis Parser constructor | ||
* @param {{returnError: Function, returnReply: Function, returnFatalError?: Function, returnBuffers: boolean, stringNumbers: boolean }} options | ||
* @constructor | ||
*/ | ||
constructor (options) { | ||
if (!options) { | ||
throw new TypeError('Options are mandatory.') | ||
} | ||
if (typeof options.returnError !== 'function' || typeof options.returnReply !== 'function') { | ||
throw new TypeError('The returnReply and returnError options have to be functions.') | ||
} | ||
this.setReturnBuffers(!!options.returnBuffers) | ||
this.setStringNumbers(!!options.stringNumbers) | ||
this.returnError = options.returnError | ||
this.returnFatalError = options.returnFatalError || options.returnError | ||
this.returnReply = options.returnReply | ||
this.reset() | ||
} | ||
/** | ||
* Reset the parser values to the initial state | ||
* | ||
* @returns {undefined} | ||
*/ | ||
reset () { | ||
this.offset = 0 | ||
} else if (this.bigStrSize === 0) { | ||
var oldLength = this.buffer.length | ||
var remainingLength = oldLength - this.offset | ||
var newBuffer = bufferAlloc(remainingLength + buffer.length) | ||
this.buffer.copy(newBuffer, 0, this.offset, oldLength) | ||
buffer.copy(newBuffer, remainingLength, 0, buffer.length) | ||
this.buffer = newBuffer | ||
this.offset = 0 | ||
if (this.arrayCache.length) { | ||
var arr = parseArrayChunks(this) | ||
if (!arr) { | ||
return | ||
} | ||
this.returnReply(arr) | ||
} | ||
} else if (this.totalChunkSize + buffer.length >= this.bigStrSize) { | ||
this.bufferCache.push(buffer) | ||
var tmp = this.optionReturnBuffers ? concatBulkBuffer(this) : concatBulkString(this) | ||
this.buffer = null | ||
this.bigStrSize = 0 | ||
this.totalChunkSize = 0 | ||
this.bufferCache = [] | ||
this.buffer = buffer | ||
if (this.arrayCache.length) { | ||
this.arrayCache[0][this.arrayPos[0]++] = tmp | ||
tmp = parseArrayChunks(this) | ||
if (!tmp) { | ||
return | ||
} | ||
this.arrayCache = [] | ||
this.arrayPos = [] | ||
} | ||
/** | ||
* Set the returnBuffers option | ||
* | ||
* @param {boolean} returnBuffers | ||
* @returns {undefined} | ||
*/ | ||
setReturnBuffers (returnBuffers) { | ||
if (typeof returnBuffers !== 'boolean') { | ||
throw new TypeError('The returnBuffers argument has to be a boolean') | ||
} | ||
this.returnReply(tmp) | ||
} else { | ||
this.bufferCache.push(buffer) | ||
this.totalChunkSize += buffer.length | ||
return | ||
this.optionReturnBuffers = returnBuffers | ||
} | ||
while (this.offset < this.buffer.length) { | ||
var offset = this.offset | ||
var type = this.buffer[this.offset++] | ||
var response = parseType(this, type) | ||
if (response === undefined) { | ||
if (!this.arrayCache.length) { | ||
this.offset = offset | ||
/** | ||
* Set the stringNumbers option | ||
* | ||
* @param {boolean} stringNumbers | ||
* @returns {undefined} | ||
*/ | ||
setStringNumbers (stringNumbers) { | ||
if (typeof stringNumbers !== 'boolean') { | ||
throw new TypeError('The stringNumbers argument has to be a boolean') | ||
} | ||
this.optionStringNumbers = stringNumbers | ||
} | ||
/** | ||
* Parse the redis buffer | ||
* @param {Buffer} buffer | ||
* @returns {undefined} | ||
*/ | ||
execute (buffer) { | ||
if (this.buffer === null) { | ||
this.buffer = buffer | ||
this.offset = 0 | ||
} else if (this.bigStrSize === 0) { | ||
const oldLength = this.buffer.length | ||
const remainingLength = oldLength - this.offset | ||
const newBuffer = Buffer.allocUnsafe(remainingLength + buffer.length) | ||
this.buffer.copy(newBuffer, 0, this.offset, oldLength) | ||
buffer.copy(newBuffer, remainingLength, 0, buffer.length) | ||
this.buffer = newBuffer | ||
this.offset = 0 | ||
if (this.arrayCache.length) { | ||
const arr = parseArrayChunks(this) | ||
if (arr === undefined) { | ||
return | ||
} | ||
this.returnReply(arr) | ||
} | ||
} else if (this.totalChunkSize + buffer.length >= this.bigStrSize) { | ||
this.bufferCache.push(buffer) | ||
var tmp = this.optionReturnBuffers ? concatBulkBuffer(this) : concatBulkString(this) | ||
this.bigStrSize = 0 | ||
this.bufferCache = [] | ||
this.buffer = buffer | ||
if (this.arrayCache.length) { | ||
this.arrayCache[0][this.arrayPos[0]++] = tmp | ||
tmp = parseArrayChunks(this) | ||
if (tmp === undefined) { | ||
return | ||
} | ||
} | ||
this.returnReply(tmp) | ||
} else { | ||
this.bufferCache.push(buffer) | ||
this.totalChunkSize += buffer.length | ||
return | ||
} | ||
if (type === 45) { | ||
this.returnError(response) | ||
} else { | ||
this.returnReply(response) | ||
while (this.offset < this.buffer.length) { | ||
const offset = this.offset | ||
const type = this.buffer[this.offset++] | ||
const response = parseType(this, type) | ||
if (response === undefined) { | ||
if (!(this.arrayCache.length || this.bufferCache.length)) { | ||
this.offset = offset | ||
} | ||
return | ||
} | ||
if (type === 45) { | ||
this.returnError(response) | ||
} else { | ||
this.returnReply(response) | ||
} | ||
} | ||
this.buffer = null | ||
} | ||
this.buffer = null | ||
} | ||
module.exports = JavascriptRedisParser |
{ | ||
"name": "redis-parser", | ||
"version": "2.6.0", | ||
"version": "3.0.0", | ||
"description": "Javascript Redis protocol (RESP) parser", | ||
@@ -30,12 +30,14 @@ "main": "index.js", | ||
"engines": { | ||
"node": ">=0.10.0" | ||
"node": ">=4" | ||
}, | ||
"dependencies": { | ||
"redis-errors": "^1.0.0" | ||
}, | ||
"devDependencies": { | ||
"benchmark": "^2.1.0", | ||
"codeclimate-test-reporter": "^0.4.0", | ||
"intercept-stdout": "^0.1.2", | ||
"hiredis": "^0.5.0", | ||
"istanbul": "^0.4.0", | ||
"standard": "^9.0.0", | ||
"mocha": "^3.1.2", | ||
"hiredis": "^0.5.0" | ||
"standard": "^10.0.0" | ||
}, | ||
@@ -42,0 +44,0 @@ "author": "Ruben Bridgewater", |
145
README.md
@@ -18,5 +18,5 @@ [![Build Status](https://travis-ci.org/NodeRedis/node-redis-parser.png?branch=master)](https://travis-ci.org/NodeRedis/node-redis-parser) | ||
```js | ||
var Parser = require('redis-parser'); | ||
const Parser = require('redis-parser'); | ||
var myParser = new Parser(options); | ||
const myParser = new Parser(options); | ||
``` | ||
@@ -35,4 +35,4 @@ | ||
* `reset()`: reset the parser to it's initial state | ||
* `setReturnBuffers(boolean)`: (JSParser only) set the returnBuffers option on/off without resetting the parser | ||
* `setStringNumbers(boolean)`: (JSParser only) set the stringNumbers option on/off without resetting the parser | ||
* `setReturnBuffers(boolean)`: set the returnBuffers option on/off without resetting the parser | ||
* `setStringNumbers(boolean)`: set the stringNumbers option on/off without resetting the parser | ||
@@ -46,3 +46,3 @@ ### Error classes | ||
All Redis errors will be returned as `ReplyErrors` while a parser error is returned as `ParserError`. | ||
All error classes are exported by the parser. | ||
All error classes can be imported by the npm `redis-errors` package. | ||
@@ -52,31 +52,34 @@ ### Example | ||
```js | ||
var Parser = require("redis-parser"); | ||
const Parser = require("redis-parser"); | ||
function Library () {} | ||
class Library { | ||
returnReply(reply) { /* ... */ } | ||
returnError(err) { /* ... */ } | ||
returnFatalError(err) { /* ... */ } | ||
Library.prototype.returnReply = function (reply) { ... } | ||
Library.prototype.returnError = function (err) { ... } | ||
Library.prototype.returnFatalError = function (err) { ... } | ||
streamHandler() { | ||
this.stream.on('data', (buffer) => { | ||
// Here the data (e.g. `Buffer.from('$5\r\nHello\r\n'`)) | ||
// is passed to the parser and the result is passed to | ||
// either function depending on the provided data. | ||
parser.execute(buffer); | ||
}); | ||
} | ||
} | ||
var lib = new Library(); | ||
const lib = new Library(); | ||
var parser = new Parser({ | ||
returnReply: function(reply) { | ||
lib.returnReply(reply); | ||
}, | ||
returnError: function(err) { | ||
lib.returnError(err); | ||
}, | ||
returnFatalError: function (err) { | ||
lib.returnFatalError(err); | ||
} | ||
const parser = new Parser({ | ||
returnReply(reply) { | ||
lib.returnReply(reply); | ||
}, | ||
returnError(err) { | ||
lib.returnError(err); | ||
}, | ||
returnFatalError(err) { | ||
lib.returnFatalError(err); | ||
} | ||
}); | ||
``` | ||
Library.prototype.streamHandler = function () { | ||
this.stream.on('data', function (buffer) { | ||
// Here the data (e.g. `new Buffer('$5\r\nHello\r\n'`)) is passed to the parser and the result is passed to either function depending on the provided data. | ||
parser.execute(buffer); | ||
}); | ||
}; | ||
``` | ||
You do not have to use the returnFatalError function. Fatal errors will be returned in the normal error function in that case. | ||
@@ -91,11 +94,11 @@ | ||
var parser = new Parser({ | ||
returnReply: function(reply) { | ||
lib.returnReply(reply); | ||
}, | ||
returnError: function(err) { | ||
lib.returnError(err); | ||
}, | ||
returnBuffers: true, // All strings are returned as Buffer e.g. <Buffer 48 65 6c 6c 6f> | ||
stringNumbers: true // All numbers are returned as String | ||
const parser = new Parser({ | ||
returnReply(reply) { | ||
lib.returnReply(reply); | ||
}, | ||
returnError(err) { | ||
lib.returnError(err); | ||
}, | ||
returnBuffers: true, // All strings are returned as Buffer e.g. <Buffer 48 65 6c 6c 6f> | ||
stringNumbers: true // All numbers are returned as String | ||
}); | ||
@@ -120,46 +123,46 @@ | ||
HIREDIS: $ multiple chunks in a bulk string x 859,880 ops/sec ±1.22% (82 runs sampled) | ||
HIREDIS BUF: $ multiple chunks in a bulk string x 608,869 ops/sec ±1.72% (85 runs sampled) | ||
JS PARSER: $ multiple chunks in a bulk string x 910,590 ops/sec ±0.87% (89 runs sampled) | ||
JS PARSER BUF: $ multiple chunks in a bulk string x 1,299,507 ops/sec ±2.18% (84 runs sampled) | ||
HIREDIS: $ multiple chunks in a bulk string x 994,387 ops/sec ±0.22% (554 runs sampled) | ||
JS PARSER: $ multiple chunks in a bulk string x 1,010,728 ops/sec ±0.28% (559 runs sampled) | ||
HIREDIS BUF: $ multiple chunks in a bulk string x 648,742 ops/sec ±0.80% (526 runs sampled) | ||
JS PARSER BUF: $ multiple chunks in a bulk string x 1,728,849 ops/sec ±0.41% (555 runs sampled) | ||
HIREDIS: + multiple chunks in a string x 1,787,203 ops/sec ±0.58% (96 runs sampled) | ||
HIREDIS BUF: + multiple chunks in a string x 943,584 ops/sec ±1.62% (87 runs sampled) | ||
JS PARSER: + multiple chunks in a string x 2,008,264 ops/sec ±1.01% (91 runs sampled) | ||
JS PARSER BUF: + multiple chunks in a string x 2,045,546 ops/sec ±0.78% (91 runs sampled) | ||
HIREDIS: + multiple chunks in a string x 1,861,132 ops/sec ±0.18% (564 runs sampled) | ||
JS PARSER: + multiple chunks in a string x 2,131,892 ops/sec ±0.31% (558 runs sampled) | ||
HIREDIS BUF: + multiple chunks in a string x 965,132 ops/sec ±0.58% (521 runs sampled) | ||
JS PARSER BUF: + multiple chunks in a string x 2,304,482 ops/sec ±0.31% (559 runs sampled) | ||
HIREDIS: $ 4mb bulk string x 310 ops/sec ±1.58% (75 runs sampled) | ||
HIREDIS BUF: $ 4mb bulk string x 471 ops/sec ±2.28% (78 runs sampled) | ||
JS PARSER: $ 4mb bulk string x 747 ops/sec ±2.43% (85 runs sampled) | ||
JS PARSER BUF: $ 4mb bulk string x 846 ops/sec ±5.52% (72 runs sampled) | ||
HIREDIS: $ 4mb bulk string x 269 ops/sec ±0.56% (452 runs sampled) | ||
JS PARSER: $ 4mb bulk string x 763 ops/sec ±0.25% (466 runs sampled) | ||
HIREDIS BUF: $ 4mb bulk string x 336 ops/sec ±0.59% (459 runs sampled) | ||
JS PARSER BUF: $ 4mb bulk string x 994 ops/sec ±0.36% (482 runs sampled) | ||
HIREDIS: + simple string x 2,324,866 ops/sec ±1.61% (90 runs sampled) | ||
HIREDIS BUF: + simple string x 1,085,823 ops/sec ±2.47% (82 runs sampled) | ||
JS PARSER: + simple string x 4,567,358 ops/sec ±1.97% (81 runs sampled) | ||
JS PARSER BUF: + simple string x 5,433,901 ops/sec ±0.66% (93 runs sampled) | ||
HIREDIS: + simple string x 2,504,305 ops/sec ±0.19% (563 runs sampled) | ||
JS PARSER: + simple string x 5,121,952 ops/sec ±0.30% (560 runs sampled) | ||
HIREDIS BUF: + simple string x 1,122,899 ops/sec ±0.52% (516 runs sampled) | ||
JS PARSER BUF: + simple string x 5,907,323 ops/sec ±0.23% (562 runs sampled) | ||
HIREDIS: : integer x 2,332,946 ops/sec ±0.47% (93 runs sampled) | ||
JS PARSER: : integer x 17,730,449 ops/sec ±0.73% (91 runs sampled) | ||
JS PARSER STR: : integer x 12,942,037 ops/sec ±0.51% (92 runs sampled) | ||
HIREDIS: : integer x 2,461,376 ops/sec ±0.14% (561 runs sampled) | ||
JS PARSER: : integer x 18,543,688 ops/sec ±0.19% (539 runs sampled) | ||
JS PARSER STR: : integer x 14,149,305 ops/sec ±0.24% (561 runs sampled) | ||
HIREDIS: : big integer x 2,012,572 ops/sec ±0.33% (93 runs sampled) | ||
JS PARSER: : big integer x 10,210,923 ops/sec ±0.94% (94 runs sampled) | ||
JS PARSER STR: : big integer x 4,453,320 ops/sec ±0.52% (94 runs sampled) | ||
HIREDIS: : big integer x 2,114,270 ops/sec ±0.15% (561 runs sampled) | ||
JS PARSER: : big integer x 10,794,439 ops/sec ±0.25% (560 runs sampled) | ||
JS PARSER STR: : big integer x 4,594,807 ops/sec ±0.24% (558 runs sampled) | ||
HIREDIS: * array x 44,479 ops/sec ±0.55% (94 runs sampled) | ||
HIREDIS BUF: * array x 14,391 ops/sec ±1.04% (86 runs sampled) | ||
JS PARSER: * array x 53,796 ops/sec ±2.08% (79 runs sampled) | ||
JS PARSER BUF: * array x 72,428 ops/sec ±0.72% (93 runs sampled) | ||
HIREDIS: * array x 45,597 ops/sec ±0.23% (565 runs sampled) | ||
JS PARSER: * array x 68,396 ops/sec ±0.30% (563 runs sampled) | ||
HIREDIS BUF: * array x 14,726 ops/sec ±0.39% (498 runs sampled) | ||
JS PARSER BUF: * array x 80,961 ops/sec ±0.25% (561 runs sampled) | ||
HIREDIS: * big nested array x 217 ops/sec ±0.97% (83 runs sampled) | ||
HIREDIS BUF: * big nested array x 255 ops/sec ±2.28% (77 runs sampled) | ||
JS PARSER: * big nested array x 242 ops/sec ±1.10% (85 runs sampled) | ||
JS PARSER BUF: * big nested array x 375 ops/sec ±1.21% (88 runs sampled) | ||
HIREDIS: * big nested array x 212 ops/sec ±0.17% (511 runs sampled) | ||
JS PARSER: * big nested array x 243 ops/sec ±0.21% (496 runs sampled) | ||
HIREDIS BUF: * big nested array x 207 ops/sec ±0.37% (430 runs sampled) | ||
JS PARSER BUF: * big nested array x 297 ops/sec ±1.10% (421 runs sampled) | ||
HIREDIS: - error x 78,821 ops/sec ±0.80% (93 runs sampled) | ||
JS PARSER: - error x 143,382 ops/sec ±0.75% (92 runs sampled) | ||
HIREDIS: - error x 168,761 ops/sec ±0.28% (559 runs sampled) | ||
JS PARSER: - error x 424,257 ops/sec ±0.28% (557 runs sampled) | ||
Platform info: | ||
Ubuntu 16.10 | ||
Node.js 7.4.0 | ||
Ubuntu 17.04 | ||
Node.js 7.10.0 | ||
Intel(R) Core(TM) i7-5600U CPU | ||
@@ -166,0 +169,0 @@ |
License Policy Violation
LicenseThis package is not allowed per your license policy. Review the package's license to ensure compliance.
Found 1 instance in 1 package
License Policy Violation
LicenseThis package is not allowed per your license policy. Review the package's license to ensure compliance.
Found 1 instance in 1 package
6
167
27717
1
7
520
+ Addedredis-errors@^1.0.0
+ Addedredis-errors@1.2.0(transitive)