enmap

Enmap - Enhanced Maps

Enhanced Maps are a data structure that can be used to store data in memory that can also be saved in a database behind the scenes. The data is synchronized to the database automatically, seamlessly, and asynchronously so it should not adversely affect your performance compared to using Maps for storage.

FAQs

Q: So what's Enmap

A: Enmaps are the Javascript Map() data structure with additional utility methods.

Q: What is "Persistent"?

A: With the use of the optional providers modules, any data added to the Enmap is stored not only in temporary memory but also backed up in a local database.

Q: How big can the Enmap be?

A: In its initial implementation, upon loading Enmap, all key/value pairs are loaded in memory. The size of the memory used is directly proportional to the size of your actual database.

Installation

To use Enmap, install it via NPM:

npm i enmap

Basic Usage

Inside your script, initialize a new Enmap:

const Enmap = require("enmap");

// Initialize an instance of Enmap
const myCollection = new Enmap();

// Adding data is simply a `set` command: 
myCollection.set("myKey", "a value");

// Getting a value is done by key 
let result = myCollection.get("myKey");

Adding Persistence

Persistence requires an additional Provider module.

Official Enmap Providers:

• Enmap-SQLite Note: Against all odds, this provider DOES support sharding!
• Enmap-Rethink Note: THE best for sharding: the only one that receives database updates on all shards (enmap-rethink 2.1.0 and higher)
• Enmap-PGSQL Note: That's shorthand for "Postgresql". Supports sharding of course.
• Enmap-Mongo Note: Yay, MongoDB! Supports sharding, duh.
• Enmap-Level Note: LevelDB does not support multiple processes or shards!

The following example uses Enmap-SQLite

// Load Enmap
const Enmap = require('enmap');

// Load EnmapSQLite
const EnmapSQLite = require('enmap-sqlite');

// Initialize the sqlite database with a table named "test"
const provider = new EnmapSQLite({ name: 'test' });

// Initialize the Enmap with the provider instance.
const myColl = new Enmap({ provider: provider });

// Persistent providers load in an **async** fashion and provide a handy defer property:

myColl.defer.then(() => {
    // all data is loaded now.
    console.log(myColl.size + "keys loaded");
});

// You can also await it if your function is async: 
(async function() {
    await myColl.defer;
    console.log(myColl.size + "keys loaded");
    // Do stuff here!
}());

// Persistent collections should be **closed** before shutdown: 
await myColl.db.close(); // or level.close() works too!

Using Enmap.multi() for multiple enmaps

To account for people that might use a large number of enmaps in the same project, I've created a new multi() method that can be used to instanciate multiple peristent enmaps together.

The method takes 3 arguments:

• An array of names for the enmaps to be created.
• A Provider (not instanciated), from any of the available ones.
• An options object containing any of the options needed to instanciate the provider. Do not add name to this, as it will use the names in the array instead.

The method returns an object where each property is a new fully-started Enmap that can be used as you would normally.

Below, an example that uses destructuring to fit all in one nice line:

const Enmap = require('enmap');
const Provider = require('enmap-mongo');
const { settings, tags, blacklist, langs } = Enmap.multi(['settings', 'tags', 'blacklist', 'langs'], Provider, { url: "mongodb://localhost:27017/enmap" });

Note that this uses a static method which means you should NOT call new Enmap() yourself, it's done within the method.

Reading and Writing Data

Reading and writing data from an enmap is as simple as from a regular map. Note that the example uses a persistent enmap, but the set and get method will work for non-persistent enmaps too. Obviously though, those values won't be persistent through reboot if you don't give a provider.

const Enmap = require('enmap');
const EnmapSQLite = require('enmap-sqlite');
// Oh look a shortcut to initializing ;)
const myColl = new Enmap({ provider: new EnmapSQLite({ name: 'test' }) });

(async function() {
    await myColl.defer;
    console.log(myColl.size + 'keys loaded');

    // Setting data is done with a key and value.
    myColl.set('simplevalue', 'this is a string');
    
    // enmap supports any **primitive** type.
    myColl.set('boolean', true);
    myColl.set('integer', 42);
    myColl.set('null', null);

    // enmap can retrieve items at any time
    const simplevalue = myColl.get('simplevalue'); // 'this is a string'
    const myboolean = myColl.get('boolean'); // true
    if(myColl.get('boolean')) console.log('yay!') // prints 'yay!' to the console.

    // You can **change** the value of a key by loading it, editing it,
    // then setting it **back** into enmap. There's no "update" function
    // it just overrides the data through the same set method: 
    myColl.set('someobject', {blah: "foo", thing: "amajig"});
    console.log(myColl.get('someobject')) // prints the object to console.

    const myObject = myColl.get('someobject'); // value is now the object with 2 properties.
    myObject.thing = "amabob"; // value of temporary object is now {blah: "foo", thing: "amabob"}
    myColl.set('someobject', myObject); // only now is it actually written correctly.
}());

Because of how javascript works, doing something like myColl.get('myobject').blah = 'meh' actually works. HOWEVER that does not trigger persistence saves even though in memory it actually does change the enmap. "fixing" this would require some "monitor" on each value which is most definitely not the sort of overhead I want to add to this code. JavaScript wasn't built for that sort of thing in mind.

UPDATING TO ENMAP 3.0

Enmap version 3.0 has some breaking changes that are important to consider when updating. These changes are:

• setProp, hasProp, getProp, removeFrom, pushIn have all been removed. These methods are now merged into their regular counterpart, which now have a new argument: "path". The path is previously what prop methods called "prop" and is the direct path to the property you want to change. Please see examples in the docs for more details on these methods. Usually you only have to remove the Prop word from the method. For setProp to set, however, pay more attention to the order of the arguments: set(key, value, path) instead of setProp(key, path, value).
• setAsync, deleteAsync were removed from Enmap. I didn't honestly see a use for them, the only time I've ever had to use them was in testing, I will figure out a different way to "wait for all changes to be written before ".
• math(), inc() and dec() were added to 2.7.3 but I didn't really advertise them. Now they're here and they're documented.

API Documentation

Enmap ⇐ Map

A enhanced Map structure with additional utility methods. Can be made persistent

Kind: global class
Extends: Map

• Enmap ⇐ Map
  • instance
    • .fetchEverything() ⇒ Map
    • .fetch(keyOrKeys) ⇒ * | Map
    • .autonum() ⇒ number
    • .changed(cb)
    • .set(key, val, path) ⇒ Map
    • .push(key, val, path, allowDupes) ⇒ Map
    • .math(key, operation, operand, path) ⇒ Map
    • .inc(key, path) ⇒ Map
    • .dec(key, path) ⇒ Map
    • .get(key, path) ⇒ *
    • .has(key, path) ⇒ boolean
    • .delete(key, path)
    • .deleteAll(bulk)
    • .remove(key, val, path) ⇒ Map
    • .array() ⇒ Array
    • .keyArray() ⇒ Array
    • .random([count]) ⇒ * | Array.<*>
    • .randomKey([count]) ⇒ * | Array.<*>
    • .findAll(prop, value) ⇒ Array
    • .find(propOrFn, [value]) ⇒ *
    • .exists(prop, value) ⇒ boolean
    • .filter(fn, [thisArg]) ⇒ Enmap
    • .filterArray(fn, [thisArg]) ⇒ Array
    • .map(fn, [thisArg]) ⇒ Array
    • .some(fn, [thisArg]) ⇒ boolean
    • .every(fn, [thisArg]) ⇒ boolean
    • .reduce(fn, [initialValue]) ⇒ *
    • .clone() ⇒ Enmap
    • .concat(...enmaps) ⇒ Enmap
    • .equals(enmap) ⇒ boolean
  • static
    • .multi(names, Provider, options) ⇒ Array.<Map>

enmap.fetchEverything() ⇒ Map

Fetches every key from the persistent enmap and loads them into the current enmap value.

Kind: instance method of Enmap
Returns: Map - The enmap containing all values.

enmap.fetch(keyOrKeys) ⇒ * | Map

Force fetch one or more key values from the enmap. If the database has changed, that new value is used.

Kind: instance method of Enmap
Returns: * | Map - A single value if requested, or a non-persistent enmap of keys if an array is requested.

Param	Type	Description
keyOrKeys	string | number	A single key or array of keys to force fetch from the enmap database.

enmap.autonum() ⇒ number

Generates an automatic numerical key for inserting a new value.

Kind: instance method of Enmap
Returns: number - The generated key number.
Example

enmap.set(enmap.autonum(), "This is a new value");

enmap.changed(cb)

Function called whenever data changes within Enmap after the initial load. Can be used to detect if another part of your code changed a value in enmap and react on it.

Kind: instance method of Enmap

Param	Type	Description
cb	function	A callback function that will be called whenever data changes in the enmap.

Example

enmap.changed((keyName, oldValue, newValue) => {
  console.log(`Value of ${key} has changed from: \n${oldValue}\nto\n${newValue});
});

enmap.set(key, val, path) ⇒ Map

Set the value in Enmap.

Kind: instance method of Enmap
Returns: Map - The Enmap.

Param	Type	Default	Description
key	string | number		Required. The key of the element to add to The Enmap. If the Enmap is persistent this value MUST be a string or number.
val	*		Required. The value of the element to add to The Enmap. If the Enmap is persistent this value MUST be stringifiable as JSON.
path	string	null	Optional. The path to the property to modify inside the value object or array. Can be a path with dot notation, such as "prop1.subprop2.subprop3"

Example

// Direct Value Examples
enmap.set('simplevalue', 'this is a string');
enmap.set('isEnmapGreat', true);
enmap.set('TheAnswer', 42);
enmap.set('IhazObjects', { color: 'black', action: 'paint', desire: true });
enmap.set('ArraysToo', [1, "two", "tree", "foor"])

// Settings Properties
enmap.set('IhazObjects', 'color', 'blue'); //modified previous object
enmap.set('ArraysToo', 2, 'three'); // changes "tree" to "three" in array.   

enmap.push(key, val, path, allowDupes) ⇒ Map

Push to an array value in Enmap.

Kind: instance method of Enmap
Returns: Map - The EnMap.

Param	Type	Default	Description
key	string | number		Required. The key of the array element to push to in Enmap. This value MUST be a string or number.
val	*		Required. The value to push to the array.
path	string	null	Optional. The path to the property to modify inside the value object or array. Can be a path with dot notation, such as "prop1.subprop2.subprop3"
allowDupes	boolean	false	Optional. Allow duplicate values in the array (default: false).

Example

// Assuming
enmap.set("simpleArray", [1, 2, 3, 4]);
enmap.set("arrayInObject", {sub: [1, 2, 3, 4]});

enmap.push("simpleArray", 5); // adds 5 at the end of the array
enmap.push("arrayInObject", "five", "sub"); adds "five" at the end of the sub array

enmap.math(key, operation, operand, path) ⇒ Map

Executes a mathematical operation on a value and saves it in the enmap.

Kind: instance method of Enmap
Returns: Map - The EnMap.

Param	Type	Default	Description
key	string | number		The enmap key on which to execute the math operation.
operation	string		Which mathematical operation to execute. Supports most math ops: =, -, *, /, %, ^, and english spelling of those operations.
operand	number		The right operand of the operation.
path	string	null	Optional. The property path to execute the operation on, if the value is an object or array.

Example

// Assuming
points.set("number", 42);
points.set("numberInObject", {sub: { anInt: 5 }});

points.math("number", "/", 2); // 21
points.math("number", "add", 5); // 26
points.math("number", "modulo", 3); // 2
points.math("numberInObject", "+", 10, "sub.anInt");

enmap.inc(key, path) ⇒ Map

Increments a key's value or property by 1. Value must be a number, or a path to a number.

Kind: instance method of Enmap
Returns: Map - The EnMap.

Param	Type	Default	Description
key	string | number		The enmap key where the value to increment is stored.
path	string	null	Optional. The property path to increment, if the value is an object or array.

Example

// Assuming
points.set("number", 42);
points.set("numberInObject", {sub: { anInt: 5 }});

points.inc("number"); // 43
points.inc("numberInObject", "sub.anInt"); // {sub: { anInt: 6 }}

enmap.dec(key, path) ⇒ Map

Decrements a key's value or property by 1. Value must be a number, or a path to a number.

Kind: instance method of Enmap
Returns: Map - The EnMap.

Param	Type	Default	Description
key	string | number		The enmap key where the value to decrement is stored.
path	string	null	Optional. The property path to decrement, if the value is an object or array.

Example

// Assuming
points.set("number", 42);
points.set("numberInObject", {sub: { anInt: 5 }});

points.dec("number"); // 41
points.dec("numberInObject", "sub.anInt"); // {sub: { anInt: 4 }}

enmap.get(key, path) ⇒ *

Retrieves a key from the enmap. If fetchAll is false, returns a promise.

Kind: instance method of Enmap
Returns: * - The value for this key.

Param	Type	Default	Description
key	string | number		The key to retrieve from the enmap.
path	string	null	Optional. The property to retrieve from the object or array. Can be a path with dot notation, such as "prop1.subprop2.subprop3"

Example

const myKeyValue = enmap.get("myKey");
console.log(myKeyValue);

const someSubValue = enmap.get("anObjectKey", "someprop.someOtherSubProp");

enmap.has(key, path) ⇒ boolean

Returns whether or not the key exists in the Enmap.

Kind: instance method of Enmap

Param	Type	Default	Description
key	string | number		Required. The key of the element to add to The Enmap or array. This value MUST be a string or number.
path	string	null	Optional. The property to verify inside the value object or array. Can be a path with dot notation, such as "prop1.subprop2.subprop3"

Example

if(enmap.has("myKey")) {
  // key is there
}

if(!enmap.has("myOtherKey", "oneProp.otherProp.SubProp")) return false;

enmap.delete(key, path)

Deletes a key in the Enmap.

Kind: instance method of Enmap

Param	Type	Default	Description
key	string | number		Required. The key of the element to delete from The Enmap.
path	string	null	Optional. The name of the property to remove from the object. Can be a path with dot notation, such as "prop1.subprop2.subprop3"

enmap.deleteAll(bulk)

Calls the delete() method on all items that have it.

Kind: instance method of Enmap

Param	Type	Default	Description
bulk	boolean	true	Optional. Defaults to True. whether to use the provider's "bulk" delete feature if it has one.

enmap.remove(key, val, path) ⇒ Map

Remove a value in an Array or Object element in Enmap. Note that this only works for values, not keys. Complex values such as objects and arrays will not be removed this way.

Kind: instance method of Enmap
Returns: Map - The EnMap.

Param	Type	Default	Description
key	string | number		Required. The key of the element to remove from in Enmap. This value MUST be a string or number.
val	*		Required. The value to remove from the array or object.
path	string	null	Optional. The name of the array property to remove from. Can be a path with dot notation, such as "prop1.subprop2.subprop3". If not presents, removes directly from the value.

enmap.array() ⇒ Array

Creates an ordered array of the values of this Enmap. The array will only be reconstructed if an item is added to or removed from the Enmap, or if you change the length of the array itself. If you don't want this caching behaviour, use Array.from(enmap.values()) instead.

Kind: instance method of Enmap

enmap.keyArray() ⇒ Array

Creates an ordered array of the keys of this Enmap The array will only be reconstructed if an item is added to or removed from the Enmap, or if you change the length of the array itself. If you don't want this caching behaviour, use Array.from(enmap.keys()) instead.

Kind: instance method of Enmap

enmap.random([count]) ⇒ * | Array.<*>

Obtains random value(s) from this Enmap. This relies on array.

Kind: instance method of Enmap
Returns: * | Array.<*> - The single value if count is undefined, or an array of values of count length

Param	Type	Description
[count]	number	Number of values to obtain randomly

enmap.randomKey([count]) ⇒ * | Array.<*>

Obtains random key(s) from this Enmap. This relies on keyArray

Kind: instance method of Enmap
Returns: * | Array.<*> - The single key if count is undefined, or an array of keys of count length

Param	Type	Description
[count]	number	Number of keys to obtain randomly

enmap.findAll(prop, value) ⇒ Array

Searches for all items where their specified property's value is identical to the given value (item[prop] === value).

Kind: instance method of Enmap

Param	Type	Description
prop	string	The property to test against
value	*	The expected value

Example

enmap.findAll('username', 'Bob');

enmap.find(propOrFn, [value]) ⇒ *

Searches for a single item where its specified property's value is identical to the given value (item[prop] === value), or the given function returns a truthy value. In the latter case, this is identical to Array.find(). All Enmap used in Discord.js are mapped using their id property, and if you want to find by id you should use the get method. See MDN for details.

Kind: instance method of Enmap

Param	Type	Description
propOrFn	string | function	The property to test against, or the function to test with
[value]	*	The expected value - only applicable and required if using a property for the first argument

Example

enmap.find('username', 'Bob');

Example

enmap.find(val => val.username === 'Bob');

enmap.exists(prop, value) ⇒ boolean

Searches for the existence of a single item where its specified property's value is identical to the given value (item[prop] === value). Do not use this to check for an item by its ID. Instead, use enmap.has(id). See MDN for details.

Kind: instance method of Enmap

Param	Type	Description
prop	string	The property to test against
value	*	The expected value

Example

if (enmap.exists('username', 'Bob')) {
 console.log('user here!');
}

enmap.filter(fn, [thisArg]) ⇒ Enmap

Identical to Array.filter(), but returns a Enmap instead of an Array.

Kind: instance method of Enmap

Param	Type	Description
fn	function	Function used to test (should return a boolean)
[thisArg]	Object	Value to use as this when executing function

enmap.filterArray(fn, [thisArg]) ⇒ Array

Identical to Array.filter().

Kind: instance method of Enmap

Param	Type	Description
fn	function	Function used to test (should return a boolean)
[thisArg]	Object	Value to use as this when executing function

enmap.map(fn, [thisArg]) ⇒ Array

Identical to Array.map().

Kind: instance method of Enmap

Param	Type	Description
fn	function	Function that produces an element of the new array, taking three arguments
[thisArg]	*	Value to use as this when executing function

enmap.some(fn, [thisArg]) ⇒ boolean

Identical to Array.some().

Kind: instance method of Enmap

Param	Type	Description
fn	function	Function used to test (should return a boolean)
[thisArg]	Object	Value to use as this when executing function

enmap.every(fn, [thisArg]) ⇒ boolean

Identical to Array.every().

Kind: instance method of Enmap

Param	Type	Description
fn	function	Function used to test (should return a boolean)
[thisArg]	Object	Value to use as this when executing function

enmap.reduce(fn, [initialValue]) ⇒ *

Identical to Array.reduce().

Kind: instance method of Enmap

Param	Type	Description
fn	function	Function used to reduce, taking four arguments; accumulator, currentValue, currentKey, and enmap
[initialValue]	*	Starting value for the accumulator

enmap.clone() ⇒ Enmap

Creates an identical shallow copy of this Enmap.

Kind: instance method of Enmap
Example

const newColl = someColl.clone();

enmap.concat(...enmaps) ⇒ Enmap

Combines this Enmap with others into a new Enmap. None of the source Enmaps are modified.

Kind: instance method of Enmap

Param	Type	Description
...enmaps	Enmap	Enmaps to merge

Example

const newColl = someColl.concat(someOtherColl, anotherColl, ohBoyAColl);

enmap.equals(enmap) ⇒ boolean

Checks if this Enmap shares identical key-value pairings with another. This is different to checking for equality using equal-signs, because the Enmaps may be different objects, but contain the same data.

Kind: instance method of Enmap
Returns: boolean - Whether the Enmaps have identical contents

Param	Type	Description
enmap	Enmap	Enmap to compare with

Enmap.multi(names, Provider, options) ⇒ Array.<Map>

Initialize multiple Enmaps easily.

Kind: static method of Enmap
Returns: Array.<Map> - An array of initialized Enmaps.

Param	Type	Description
names	Array.<string>	Array of strings. Each array entry will create a separate enmap with that name.
Provider	EnmapProvider	Valid EnmapProvider object.
options	Object	Options object to pass to the provider. See provider documentation for its options.

Example

// Using local variables and the mongodb provider.
const Enmap = require('enmap');
const Provider = require('enmap-mongo');
const { settings, tags, blacklist } = Enmap.multi(['settings', 'tags', 'blacklist'], Provider, { url: "some connection URL here" });

// Attaching to an existing object (for instance some API's client)
const Enmap = require("enmap");
const Provider = require("enmap-mongo");
Object.assign(client, Enmap.multi(["settings", "tags", "blacklist"], Provider, { url: "some connection URL here" }));