A NodeJS Server for LokiJS
Sloki is a nodejs server which embed LokiJS, a blazing fast in-memory documents database. Sloki help to make LokiJS scalable : you can now have multiple processes speaking with LokiJS through Sloki.
A possible architecture using sloki :
+----------------------------+ TCP / Binary +-----------------------------------+
| NodeJS app worker #1 |<------------------->| |
+----------------------------+ | Sloki |
| |
+----------------------------+ TCP / Dinary | +-------------------------+ |
| NodeJS app worker #2 |<------------------->| | | |
+----------------------------+ | | | |
| | LokiJS | |
+----------------------------+ TCP / JSONRPC | | fast in-memory database | |
| go/php/python/C/whatever |<------------------->| | | |
+----------------------------+ | | | |
| | | |
+----------------------------+ TCP / Binary | +-------------------------+ |
| sloki-cli |<------------------->| |
+----------------------------+ +-----------------------------------+
The goal is to have a complete in-memory database, supporting documents (mongodb like) AND key/value stores (like redis like).
For documents, lokiJS is used.
Key/value stores are not implemented at the moment.
For moment, only TCP transport is implemented. The advantage of TCP vs HTTP API is that the connection is persistent (i.e more fast). Websockets should be implemented before HTTP API.
By default, Sloki listens on the following ports:
| Port | Transport | TLS | Protocol |
|---|---|---|---|
| 6370 | TCP | NO | Binary |
| 6371 | TCP | YES | Binary |
| 6372 | TCP | NO | JSONRPC |
| 6373 | TCP | YES | JSONRPC |
| 6374 | TCP | NO | Dinary |
| 6375 | TCP | YES | Dinary (fastest) |
If somebody have an idea why TLS over TCP is fastest than raw TCP, i'd like to know .. :)
You will need a client to speak with sloki.
The binary protocol has been made with performance in mind. Payloads looks like JSONRPC, but it's not.
REQUEST | RESPONSE
------------------------------------------- | --------------------------------------
{ | {
"m":"myMethod", | "r":true,
"p":["foo","bar"], | "id":"operation-uniq-id"
"id":"operation-uniq-id" | }
} |
jsonrpc version attribute, method become m, params become p, result become r)The JSONRPC protocol has been chosen for interoperability.
REQUEST | RESPONSE
------------------------------------------- | --------------------------------------
{ | {
"jsonrpc":"2.0", | "jsonrpc":"2.0"
"method":"myMethod", | "result":true,
"params":["foo","bar"], | "id":"operation-uniq-id"
"id":"operation-uniq-id" | }
} |
It's not a typo. Dinary use 2 Binary clients, one socket for requests, the other one for responses. This is the fastest protocol, the one by default.
The underling protocol is the same as the Binary one.
npm install -g sloki
sloki
sloki --help
=======================================================================
Sloki - a NodeJS Server for LokyJS
=======================================================================
Environment variable Default
SLOKI_TCP_BINARY_ENABLE true
SLOKI_TCP_BINARY_PORT 6370
SLOKI_TCP_BINARY_HOST localhost
SLOKI_TCP_BINARY_MAX_CLIENTS 64
SLOKI_TLS_BINARY_ENABLE true
SLOKI_TLS_BINARY_PORT 6371
SLOKI_TLS_BINARY_HOST localhost
SLOKI_TLS_BINARY_MAX_CLIENTS 64
SLOKI_TCP_JSONRPC_ENABLE true
SLOKI_TCP_JSONRPC_PORT 6372
SLOKI_TCP_JSONRPC_HOST localhost
SLOKI_TCP_JSONRPC_MAX_CLIENTS 64
SLOKI_TLS_JSONRPC_ENABLE true
SLOKI_TLS_JSONRPC_PORT 6373
SLOKI_TLS_JSONRPC_HOST localhost
SLOKI_TLS_JSONRPC_MAX_CLIENTS 64
SLOKI_TCP_DINARY_ENABLE true
SLOKI_TCP_DINARY_PORT 6374
SLOKI_TCP_DINARY_HOST localhost
SLOKI_TCP_DINARY_MAX_CLIENTS 64
SLOKI_TLS_DINARY_ENABLE true
SLOKI_TLS_DINARY_PORT 6375
SLOKI_TLS_DINARY_HOST localhost
SLOKI_TLS_DINARY_MAX_CLIENTS 64
SLOKI_DIR /home/franck/.sloki
SLOKI_SHOW_OPS_INTERVAL 0
SLOKI_GC_INTERVAL 3600000
SLOKI_MEM_LIMIT 26094 Mb
-----------------------------------------------------------------------
Command Line Options Default
--tcp-binary-enable true
--tcp-binary-port 6370
--tcp-binary-host localhost
--tcp-binary-max-clients 64
--tls-binary-enable true
--tls-binary-port 6371
--tls-binary-host localhost
--tls-binary-max-clients 64
--tcp-jsonrpc-enable true
--tcp-jsonrpc-port 6372
--tcp-jsonrpc-host localhost
--tcp-jsonrpc-max-clients 64
--tls-jsonrpc-enable true
--tls-jsonrpc-port 6373
--tls-jsonrpc-host localhost
--tls-jsonrpc-max-clients 64
--tcp-dinary-enable undefined
--tcp-dinary-port 6374
--tcp-dinary-host localhost
--tcp-dinary-max-clients 64
--tls-dinary-enable true
--tls-dinary-port 6375
--tls-dinary-host localhost
--tls-dinary-max-clients 64
--dir /home/franck/.sloki
--show-ops-interval 0
--gc-interval 3600000
--mem-limit 26094 Mb
-----------------------------------------------------------------------
Examples:
$ sloki # will use defaults
$ sloki --tcp-binary-port=6370 --tcp-binary-host=localhost
See https://github.com/sloki-project/sloki-node-client
The client will load every methods that sloki server have.
> npm run bench
#################################################################################
# Benchmark suite using sloki v0.0.8 Intel® Core™ i7-6820HQ 2.70Ghz
#################################################################################
# x64 | 8 CPU(s) | linux (4.4.0-43-Microsoft Linux) | node v11.10.0
#################################################################################
> client connected (binary)
> client connected (binarys)
> client connected (jsonrpc)
> client connected (jsonrpcs)
> client connected (dinary)
> client connected (dinarys)
>>>>> test insert#nocallback
> run insert#nocallback@binary
> run insert#nocallback@binarys
> run insert#nocallback@jsonrpc
> run insert#nocallback@jsonrpcs
> run insert#nocallback@dinary
> run insert#nocallback@dinarys
>>>>> test insert#callback.fullDocument
> run insert#callback.fullDocument@binary
> run insert#callback.fullDocument@binarys
> run insert#callback.fullDocument@jsonrpc
> run insert#callback.fullDocument@jsonrpcs
> run insert#callback.fullDocument@dinary
> run insert#callback.fullDocument@dinarys
>>>>> test insert#callback.sret.01
> run insert#callback.sret.01@binary
> run insert#callback.sret.01@binarys
> run insert#callback.sret.01@jsonrpc
> run insert#callback.sret.01@jsonrpcs
> run insert#callback.sret.01@dinary
> run insert#callback.sret.01@dinarys
>>>>> gc done (rss before 206 MB, after 194 MB)
> client disconnected (binary)
> client disconnected (binarys)
> client disconnected (jsonrpc)
> client disconnected (jsonrpcs)
> client disconnected (dinary)
> client disconnected (dinarys)
# --------------------------------------------------------------------------------
# Test | Operations | ops/sec | exec time
# --------------------------------------------------------------------------------
# insert#nocallback@binary | 20000 | 12462 | 1.6s
# insert#nocallback@binarys | 20000 | 314297 | 64ms
# insert#nocallback@jsonrpc | 20000 | 19875 | 1s
# insert#nocallback@jsonrpcs | 20000 | 359086 | 56ms
# insert#nocallback@dinary | 20000 | 14190 | 1.4s
# insert#nocallback@dinarys | 20000 | 939276 | 22ms
# insert#callback.fullDocument@binary | 20000 | 11682 | 1.7s
# insert#callback.fullDocument@binarys | 20000 | 9542 | 2.1s
# insert#callback.fullDocument@jsonrpc | 20000 | 17443 | 1.1s
# insert#callback.fullDocument@jsonrpcs | 20000 | 17136 | 1.2s
# insert#callback.fullDocument@dinary | 20000 | 11089 | 1.8s
# insert#callback.fullDocument@dinarys | 20000 | 26836 | 746ms
# insert#callback.sret.01@binary | 20000 | 11939 | 1.7s
# insert#callback.sret.01@binarys | 20000 | 27308 | 733ms
# insert#callback.sret.01@jsonrpc | 20000 | 17117 | 1.2s
# insert#callback.sret.01@jsonrpcs | 20000 | 17031 | 1.2s
# insert#callback.sret.01@dinary | 20000 | 11184 | 1.8s
# insert#callback.sret.01@dinarys | 20000 | 26900 | 744ms
The winner is dinary protocol !
Not usable yet.
FAQs
A NodeJS server for LokiJS
We found that sloki demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
/Security News
Two npm packages masquerading as WhatsApp developer libraries include a kill switch that deletes all files if the phone number isn’t whitelisted.
Research
/Security News
Socket uncovered 11 malicious Go packages using obfuscated loaders to fetch and execute second-stage payloads via C2 domains.
Security News
TC39 advances 11 JavaScript proposals, with two moving to Stage 4, bringing better math, binary APIs, and more features one step closer to the ECMAScript spec.