tcp-log-server

Serve an append-only log over TCP.

• Uses a dead-simple, JSON-based protocol.
• Stores entries as content-addressed blobs.
• Persists to disk by default.

npm install --global tcp-log-server
tcp-log-server

Logs

The server uses pino logging. To improve console log output:

npm install --global pino
tcp-log-server | tee server.log | pino

Environment

Configure tcp-log-server with environment variables:

• PORT for TCP
• BLOBS directory for entry JSON files, named by hash
• LEVELDB directory for LevelDB data files

Node.js

The package exports a factory function. Given a pino log, a LevelUP instance, an abstract-blob-store, and an EventEmitter, it returns a TCP connection handler function suitable for net.createServer(handler). Any LevelDOWN and abstract-blob-store will do. Store log entires or blobs in memory, in a remote store, or in whatever combination you like.

There is one caveat: The storage back-end for the LevelUP must support snapshots. level-party and a few other helper packages prevent snapshotting.

Protocol

The server accepts TCP connections. It enables keep-alive on each socket. All messages are newline-delimited JSON objects. tcp-log-client provides a high-level interface.

Clients can send:

Read

{"from":1}

On receipt, the server will begin sending log entries with indices greater than or equal to from. (The first index is 1.) Each entry will be sent like:

{"index":"1","entry":{"some":"entry"}}

Servers may report errors reading specific log entries:

{"index":45,"error":"some-error"}

When the server has sent all entries in the log as of the time the read message was received, it will send:

{"current":true}

Write

{"id":"some-id","entry":{"arbitrary":"data"}}

require('uuid').v4(), with the uuid package, is an easy way to generate id strings.

Once successfully appended to the log, the server will confirm the index of the newly appended entry.

{"id":"some-id","index":44}

If there is an error, the server will instead respond:

{"id":"some-id-string","error":"error-string"}