jsr

Table of Contents

• Redis Toolkit
  • Install
  • Usage
  • Use Cases
  • Performance
    • Machine
    • Memory
    • Example
  • Story
  • Modules
  • Notes
    • Fork
    • Pattern
  • Developer
    • Import
    • Documentation
    • Readme
  • License

Redis Toolkit

Install

npm i -g jsr

Requires node and npm.

Usage

Start a server on the default port 9736:

jsrs

Start a client:

jsrc

Or if you prefer redis-cli -p 9736.

Play.

Use Cases

If performance is critical or you have limited resources just use redis. However there are some scenarios where this toolkit could be beneficial, primarily the scenario described in the story (for an example of how to do this see the ps test spec):

• Custom commands, extend the classes in exec and load them at boot time.
• Distributed node applications that can share state (cluster IPC is synchronous).
• Inspect the state of an application at runtime, just call listen() on the server and connect with redis-cli(1), jsrc(1) or telnet(1).
• HTTP interface to redis.

Performance

Excellent C code is always going to perform much better than Javascript running in V8, however you can get a boost in performance by installing hiredis. If hiredis is available anywhere in the node module lookup hierarchy it will be used for decoding by the resp library.

It is not installed by default to gain a better idea of the performance difference.

Machine

Processor Name:         Intel Core i7
Processor Speed:        2.66 GHz
Number Of Processors:   1
Total Number Of Cores:  2
L2 Cache (per core):    256 KB
L3 Cache:               4 MB
Memory:                 8 GB

Memory

On the machine indicated above starting redis-server consumes about 2MB of memory with no activity, starting jsrs will result in about 28MB of memory consumption with no activity due to the V8 allocations.

Example

redis-server
time redis-cli -r 1000 set key value

Yields:

redis-cli -r 1000 set key value  0.01s user 0.02s system 38% cpu 0.072 total

jsrs
time redis-cli -p 9736 -r 1000 set key value

Yields:

redis-cli -p 9736 -r 1000 set key value  0.01s user 0.02s system 11% cpu 0.282 total

Story

Whilst designing a server application with a requirement to run as a single-process, using the cluster module and distributed, redis was the clear choice for multi-process execution that needed to share connection state between the processes.

When running as a single-process it made more sense to keep connection state in the process rather than have additional TCP and RESP overhead. The result was working code, but the implementation didn't feel right. The multi-process code was operating on key value pairs whilst the single-process code was a nested hierarchy of objects and arrays. The thought occured that the single-process code could also use a key value store and mimic the redis client API, this would make the code much simpler and reduce the likelihood of error. It would also enable easily comparing performance of the single-process server when storing data in-memory and in redis. The idea being that changing a single line of code would switch the storage layer.

The multi-process version was already using transactions, sets and various other redis features which meant the in-memory version needed to expose that functionality in order to be compatible.

Conceived to solve this particular architectural problem the toolkit quickly evolved into a complete redis clone inspired by a desire to learn more about redis.

Modules

• server - Server component.
• client - Client component.
• store - Memory key value store.
• resp - Streaming RESP protocol library.
• conf - Configuration file parser and writer.
• rdb - Encoder and decoder for RDBv6.
• validate - Command validation library.
• definition - Command definition class.
• exec - Abstract command classes.
• loader - Command loader.
• util - Utility functions.
• log - Server log component.
• pattern - Pattern matching library.
• constants - Constants used across all modules.
• error - Error classes and static error definitions.
• script - Script execution.
• format - Format and string parsing classes.
• persistence - Database persistence.

Notes

Fork

The configuration parser supports all the redis configuration directives but some do not apply, specifically the fork and fsync directives and statistics.

The node fork implementation is not equivalent to fork(2), the child process does not have access to the parent process' memory, therefore where redis would use fork(2) the code will use setImmediate to break lots of work into chunks and not block the event loop or starve I/O allowing client connections to still be served.

Pattern

When a command accepts a pattern in order to match a special character (*?[]) two backslashes are required due to Javascript treating a backslash within a string as an escape character, eg: question\\? to match the question mark literally.

Developer

Import

To update the commands documentation run (requires curl(1) and perl(1)):

npm run import

Documentation

To generate all documentation:

npm run docs

Readme

To build the readme file from the partial definitions (requires mdp):

npm run readme

License

Everything is MIT. Read the license if you feel inclined.

Generated by mdp(1).