jsonschema-key-compression

jsonschema-key-compression

Compress json-data based on it's json-schema while still having valid json. It works by compressing long attribute-names into smaller ones and backwards.

For example this:

{
    "firstName": "alice",
    "lastName": "wonder",
    "registerDate": "2019-06-01",
    "country": "de",
    "active": true,
    "eyeColor": "brown"
}

becomes this:

{
    "|a": "alice",
    "|b": "wonder",
    "|c": "2019-06-01",
    "|d": "de",
    "|e": true,
    "|f": "brown"
}

The compressed version only needs 85 chars while the non-compressed version needs 123 chars. So by storing the compressed version, you can store up to 30% more data.

You should use this when

• you want to save storage space in an NoSQL-database but still want to have valid json-data
• you transmit many objects in many small requests over the network so that gzip cannot be efficient
• you want to store json-data inside of the browser-storage (indexedDB or localstorage) and you reach the storage limit

You should NOT use this when

• you send many objects in a single request, you should rely on gzip instead
• you do not want to still have valid json-data, you should use protobuf instead
• you have schema-less data

Usage

Install

npm install jsonschema-key-compression --save

createCompressionTable

Creates a compression-table from the json-schema.

import {
    createCompressionTable
} from 'jsonschema-key-compression';
const compressionTable = createCompressionTable(jsonSchema);

compressObject

Compress a json-object based on it's schema.

import {
    compressObject
} from 'jsonschema-key-compression';
const compressedObject = createCompressionTable(
    compressionTable,
    jsonObject
);

decompressObject

Decompress a compressed object.

import {
    decompressObject
} from 'jsonschema-key-compression';
const jsonObject = decompressObject(
    compressionTable,
    compressionTable
);

compressedPath

Transform a chain of json-attributes into it's compresed format.

import {
    compressedPath
} from 'jsonschema-key-compression';
const compressed = compressedPath(
    compressionTable,
    'whateverNested.firstName'
); // > '|a.|b'

decompressedPath

Decompress a compressed path.

import {
    decompressedPath
} from 'jsonschema-key-compression';
const decompressed = decompressedPath(
    compressionTable,
    '|a.|b' // from compressedPath
);

compressQuery

Compress a mango-query so that it can run over a NoSQL-Database that has stored compressed documents.

import {
    compressQuery
} from 'jsonschema-key-compression';
const decompressed = compressQuery(
    compressionTable,
    {
        selector: {
            active: {
                $eq: true
            }
        },
        skip: 1,
        limit: 1,
        fields: [
        'id',
        'name'
        ],
        sort: [
            'name'
        ]
    }
);