enmap

Enmap - Enhanced Maps

Enhanced Maps are a data structure that can be used to store data in memory that can also be saved in a database behind the scenes. The data is synchronized to the database automatically, seamlessly, and asynchronously so it should not adversely affect your performance compared to using Maps for storage.

FAQs

Q: So what's Enmap

A: Enmaps are the Javascript Map() data structure with additional utility methods.

Q: What is "Persistent"?

A: With the use of the optional providers modules, any data added to the Enmap is stored not only in temporary memory but also backed up in a local database.

Q: How big can the Enmap be?

A: In its initial implementation, upon loading Enmap, all key/value pairs are loaded in memory. The size of the memory used is directly proportional to the size of your actual database.

Future versions will have ways to load partial or temporary values, etc.

Installation

To use Enmap, install it via NPM:

npm i enmap

Basic Usage

Inside your script, initialize a new Enmap:

const Enmap = require("enmap");

// Initialize an instance of Enmap
const myCollection = new Enmap();

// Adding data is simply a `set` command: 
myCollection.set("myKey", "a value");

// Getting a value is done by key 
let result = myCollection.get("myKey");

Adding Persistence

Persistence requires an additional Provider module.

Official Enmap Providers:

• Enmap-SQLite Note: Against all odds, this provider DOES support sharding!
• Enmap-Rethink Note: THE best for sharding: the only one that receives database updates on all shards (enmap-rethink 2.1.0 and higher)
• Enmap-PGSQL Note: That's shorthand for "Postgresql". Supports sharding of course.
• Enmap-Mongo Note: Yay, MongoDB! Supports sharding, duh.
• Enmap-Level Note: LevelDB does not support multiple processes or shards!

The following example uses Enmap-SQLite

// Load Enmap
const Enmap = require('enmap');

// Load EnmapSQLite
const EnmapSQLite = require('enmap-sqlite');

// Initialize the sqlite database with a table named "test"
const provider = new EnmapSQLite({ name: 'test' });

// Initialize the Enmap with the provider instance.
const myColl = new Enmap({ provider: provider });

// Persistent providers load in an **async** fashion and provide a handy defer property:

myColl.defer.then(() => {
    // all data is loaded now.
    console.log(myColl.size + "keys loaded");
});

// You can also await it if your function is async: 
(async function() {
    await myColl.defer;
    console.log(myColl.size + "keys loaded");
    // Do stuff here!
}());

// Persistent collections should be **closed** before shutdown: 
await myColl.db.close(); // or level.close() works too!

Using Enmap.multi() for multiple enmaps

To account for people that might use a large number of enmaps in the same project, I've created a new multi() method that can be used to instanciate multiple peristent enmaps together.

The method takes 3 arguments:

• An array of names for the enmaps to be created.
• A Provider (not instanciated), from any of the available ones.
• An options object containing any of the options needed to instanciate the provider. Do not add name to this, as it will use the names in the array instead.

The method returns an object where each property is a new fully-started Enmap that can be used as you would normally.

Below, an example that uses destructuring to fit all in one nice line:

const Enmap = require('enmap');
const Provider = require('enmap-mongo');
const { settings, tags, blacklist, langs } = Enmap.multi(['settings', 'tags', 'blacklist', 'langs'], Provider, { url: "mongodb://localhost:27017/enmap" });

Note that this uses a static method which means you should NOT call new Enmap() yourself, it's done within the method.

Reading and Writing Data

Reading and writing data from an enmap is as simple as from a regular map. Note that the example uses a persistent enmap, but the set and get method will work for non-persistent enmaps too. Obviously though, those values won't be persistent through reboot if you don't give a provider.

const Enmap = require('enmap');
const EnmapSQLite = require('enmap-sqlite');
// Oh look a shortcut to initializing ;)
const myColl = new Enmap({ provider: new EnmapSQLite({ name: 'test' }) });

(async function() {
    await myColl.defer;
    console.log(myColl.size + 'keys loaded');

    // Setting data is done with a key and value.
    myColl.set('simplevalue', 'this is a string');
    
    // enmap supports any **primitive** type.
    myColl.set('boolean', true);
    myColl.set('integer', 42);
    myColl.set('null', null);

    // enmap can retrieve items at any time
    const simplevalue = myColl.get('simplevalue'); // 'this is a string'
    const myboolean = myColl.get('boolean'); // true
    if(myColl.get('boolean')) console.log('yay!') // prints 'yay!' to the console.

    // You can **change** the value of a key by loading it, editing it,
    // then setting it **back** into enmap. There's no "update" function
    // it just overrides the data through the same set method: 
    myColl.set('someobject', {blah: "foo", thing: "amajig"});
    console.log(myColl.get('someobject')) // prints the object to console.

    const myObject = myColl.get('someobject'); // value is now the object with 2 properties.
    myObject.thing = "amabob"; // value of temporary object is now {blah: "foo", thing: "amabob"}
    myColl.set('someobject', myObject); // only now is it actually written correctly.
}());

Because of how javascript works, doing something like myColl.get('myobject').blah = 'meh' actually works. HOWEVER that does not trigger persistence saves even though in memory it actually does change the enmap. "fixing" this would require some "monitor" on each value which is most definitely not the sort of overhead I want to add to this code. JavaScript wasn't built for that sort of thing in mind.

API Documentation

Enmap ⇒ Map

A enhanced Map structure with additional utility methods. Can be made persistent

Kind: global class
Extends: Map

• Enmap ⇐ Map
  • instance
    • .fetchEverything() ⇒ Map
    • .fetch(keyOrKeys) ⇒ * | Map
    • .autonum() ⇒ number
    • .changed(cb)
    • .set(key, val) ⇒ Map
    • .setAsync(key, val) ⇒ Promise.<Map>
    • .setProp(key, prop, val, save) ⇒ Map
    • .push(key, val, allowDupes) ⇒ Map
    • .pushIn(key, prop, val, allowDupes) ⇒ Map
    • .get(key) ⇒ * | Promise.<*>
    • .getProp(key, prop) ⇒ *
    • .getHighestAutonum(start) ⇒ Integer
    • .has(key) ⇒ Promise.<boolean>
    • .hasProp(key, prop) ⇒ boolean
    • .delete(key, bulk)
    • .deleteAsync(key, bulk)
    • .deleteAll(bulk)
    • .deleteAllAsync(bulk) ⇒ Promise
    • .remove(key, val, allowDupes) ⇒ Map
    • .removeFrom(key, prop, val) ⇒ Map
    • .deleteProp(key, prop) ⇒ Promise.<Enmap> | Enmap
    • .array() ⇒ Array
    • .keyArray() ⇒ Array
    • .random([count]) ⇒ * | Array.<*>
    • .randomKey([count]) ⇒ * | Array.<*>
    • .findAll(prop, value) ⇒ Array
    • .find(propOrFn, [value]) ⇒ *
    • .exists(prop, value) ⇒ boolean
    • .filter(fn, [thisArg]) ⇒ Enmap
    • .filterArray(fn, [thisArg]) ⇒ Array
    • .map(fn, [thisArg]) ⇒ Array
    • .some(fn, [thisArg]) ⇒ boolean
    • .every(fn, [thisArg]) ⇒ boolean
    • .reduce(fn, [initialValue]) ⇒ *
    • .clone() ⇒ Enmap
    • .concat(...enmaps) ⇒ Enmap
    • .equals(enmap) ⇒ boolean
  • static
    • .multi(names, Provider, options) ⇒ Array.<Map>

enmap.fetchEverything() ⇒ Map

Fetches every key from the persistent enmap and loads them into the current enmap value.

Kind: instance method of Enmap
Returns: Map - The enmap containing all values.

enmap.fetch(keyOrKeys) ⇒ * | Map

Force fetch one or more key values from the enmap. If the database has changed, that new value is used.

Kind: instance method of Enmap
Returns: * | Map - A single value if requested, or a non-persistent enmap of keys if an array is requested.

Param	Type	Description
keyOrKeys	string | number	A single key or array of keys to force fetch from the enmap database.

enmap.autonum() ⇒ number

Generates an automatic numerical key for inserting a new value.

Kind: instance method of Enmap
Returns: number - The generated key number.

enmap.changed(cb)

Function called whenever data changes within Enmap after the initial load. Can be used to detect if another part of your code changed a value in enmap and react on it.

Kind: instance method of Enmap

Param	Type	Description
cb	function	A callback function that will be called whenever data changes in the enmap.

Example

enmap.changed((keyName, oldValue, newValue) => {
  console.log(`Value of ${key} has changed from: \n${oldValue}\nto\n${newValue});
})

enmap.set(key, val) ⇒ Map

Set the value in Enmap.

Kind: instance method of Enmap
Returns: Map - The Enmap.

Param	Type	Description
key	string | number	Required. The key of the element to add to The Enmap. If the Enmap is persistent this value MUST be a string or number.
val	*	Required. The value of the element to add to The Enmap. If the Enmap is persistent this value MUST be stringifiable as JSON.

Example

enmap.set('simplevalue', 'this is a string');
enmap.set('isEnmapGreat', true);
enmap.set('TheAnswer', 42);
enmap.set('IhazObjects', { color: 'black', action: 'paint', desire: true });

enmap.setAsync(key, val) ⇒ Promise.<Map>

Set the value in Enmap, but returns a promise that resolves once writte to the database. Useless on non-persistent Enmaps.

Kind: instance method of Enmap
Returns: Promise.<Map> - The Enmap.

Param	Type	Description
key	string | number	Required. The key of the element to add to The Enmap. If the Enmap is persistent this value MUST be a string or number.
val	*	Required. The value of the element to add to The Enmap. If the Enmap is persistent this value MUST be stringifiable as JSON.

enmap.setProp(key, prop, val, save) ⇒ Map

Modify the property of a value inside the enmap, if the value is an object or array. This is a shortcut to loading the key, changing the value, and setting it back.

Kind: instance method of Enmap
Returns: Map - The EnMap.

Param	Type	Description
key	string | number	Required. The key of the element to add to The Enmap or array. This value MUST be a string or number.
prop	*	Required. The property to modify inside the value object or array.
val	*	Required. The value to apply to the specified property.
save	boolean	Optional. Whether to save to persistent DB (used as false in init)

enmap.push(key, val, allowDupes) ⇒ Map

Push to an array value in Enmap.

Kind: instance method of Enmap
Returns: Map - The EnMap.

Param	Type	Default	Description
key	string | number		Required. The key of the array element to push to in Enmap. This value MUST be a string or number.
val	*		Required. The value to push to the array.
allowDupes	boolean	false	Allow duplicate values in the array (default: false).

enmap.pushIn(key, prop, val, allowDupes) ⇒ Map

Push to an array element inside an Object or Array element in Enmap.

Kind: instance method of Enmap
Returns: Map - The EnMap.

Param	Type	Default	Description
key	string | number		Required. The key of the element. This value MUST be a string or number.
prop	*		Required. The name of the array property to push to.
val	*		Required. The value push to the array property.
allowDupes	boolean	false	Allow duplicate values in the array (default: false).

enmap.get(key) ⇒ * | Promise.<*>

Retrieves a key from the enmap. If fetchAll is false, returns a promise.

Kind: instance method of Enmap
Returns: * | Promise.<*> - The value or a promise containing the value.

Param	Type	Description
key	string | number	The key to retrieve from the enmap.

Example

const myKeyValue = enmap.get("myKey");
console.log(myKeyValue);

enmap.getProp(key, prop) ⇒ *

Returns the specific property within a stored value. If the key does not exist or the value is not an object, throws an error.

Kind: instance method of Enmap
Returns: * - The value of the property obtained.

Param	Type	Description
key	string | number	Required. The key of the element to get from The Enmap.
prop	*	Required. The property to retrieve from the object or array.

enmap.getHighestAutonum(start) ⇒ Integer

Internal method used by autonum(). Loops on incremental numerical values until it finds a free key of that value in the Enamp.

Kind: instance method of Enmap
Returns: Integer - The first non-existant value found.

Param	Type	Default	Description
start	Integer	0	The starting value to look for.

enmap.has(key) ⇒ Promise.<boolean>

Returns whether or not the key exists in the Enmap.

Kind: instance method of Enmap

Param	Type	Description
key	string | number	Required. The key of the element to add to The Enmap or array. This value MUST be a string or number.

enmap.hasProp(key, prop) ⇒ boolean

Returns whether or not the property exists within an object or array value in enmap.

Kind: instance method of Enmap
Returns: boolean - Whether the property exists.

Param	Type	Description
key	string | number	Required. The key of the element to check in the Enmap or array.
prop	*	Required. The property to verify inside the value object or array.

enmap.delete(key, bulk)

Deletes a key in the Enmap.

Kind: instance method of Enmap

Param	Type	Description
key	string | number	Required. The key of the element to delete from The Enmap.
bulk	boolean	Internal property used by the purge method.

enmap.deleteAsync(key, bulk)

Kind: instance method of Enmap

Param	Type	Description
key	string | number	Required. The key of the element to delete from The Enmap.
bulk	boolean	Internal property used by the purge method.

enmap.deleteAll(bulk)

Calls the delete() method on all items that have it.

Kind: instance method of Enmap

Param	Type	Default	Description
bulk	boolean	true	Optional. Defaults to True. whether to use the provider's "bulk" delete feature if it has one.

enmap.deleteAllAsync(bulk) ⇒ Promise

Calls the delete() method on all items that have it.

Kind: instance method of Enmap
Returns: Promise - Returns a promise that is resolved when the database is cleared.

Param	Type	Default	Description
bulk	boolean	true	Optional. Defaults to True. whether to use the provider's "bulk" delete feature if it has one.

enmap.remove(key, val, allowDupes) ⇒ Map

Remove a value in an Array or Object element in Enmap. Note that this only works for values, not keys. Complex values such as objects and arrays will not be removed this way.

Kind: instance method of Enmap
Returns: Map - The EnMap.

Param	Type	Description
key	string | number	Required. The key of the element to remove from in Enmap. This value MUST be a string or number.
val	*	Required. The value to remove from the array or object.
allowDupes	boolean	Allow duplicate values in the array (default: false).

enmap.removeFrom(key, prop, val) ⇒ Map

Remove a value from an Array or Object property inside an Array or Object element in Enmap. Confusing? Sure is.

Kind: instance method of Enmap
Returns: Map - The EnMap.

Param	Type	Description
key	string | number	Required. The key of the element. This value MUST be a string or number.
prop	*	Required. The name of the array property to remove from.
val	*	Required. The value to remove from the array property.

enmap.deleteProp(key, prop) ⇒ Promise.<Enmap> | Enmap

Delete a property from an object or array value in Enmap.

Kind: instance method of Enmap
Returns: Promise.<Enmap> | Enmap - If fetchAll is true, return the Enmap. Otherwise return a promise containing the Enmap.

Param	Type	Description
key	string | number	Required. The key of the element to delete the property from in Enmap.
prop	*	Required. The name of the property to remove from the object.

enmap.array() ⇒ Array

Creates an ordered array of the values of this Enmap. The array will only be reconstructed if an item is added to or removed from the Enmap, or if you change the length of the array itself. If you don't want this caching behaviour, use Array.from(enmap.values()) instead.

Kind: instance method of Enmap

enmap.keyArray() ⇒ Array

Creates an ordered array of the keys of this Enmap The array will only be reconstructed if an item is added to or removed from the Enmap, or if you change the length of the array itself. If you don't want this caching behaviour, use Array.from(enmap.keys()) instead.

Kind: instance method of Enmap

enmap.random([count]) ⇒ * | Array.<*>

Obtains random value(s) from this Enmap. This relies on array.

Kind: instance method of Enmap
Returns: * | Array.<*> - The single value if count is undefined, or an array of values of count length

Param	Type	Description
[count]	number	Number of values to obtain randomly

enmap.randomKey([count]) ⇒ * | Array.<*>

Obtains random key(s) from this Enmap. This relies on keyArray

Kind: instance method of Enmap
Returns: * | Array.<*> - The single key if count is undefined, or an array of keys of count length

Param	Type	Description
[count]	number	Number of keys to obtain randomly

enmap.findAll(prop, value) ⇒ Array

Searches for all items where their specified property's value is identical to the given value (item[prop] === value).

Kind: instance method of Enmap

Param	Type	Description
prop	string	The property to test against
value	*	The expected value

Example

enmap.findAll('username', 'Bob');

enmap.find(propOrFn, [value]) ⇒ *

Searches for a single item where its specified property's value is identical to the given value (item[prop] === value), or the given function returns a truthy value. In the latter case, this is identical to Array.find(). All Enmap used in Discord.js are mapped using their id property, and if you want to find by id you should use the get method. See MDN for details.

Kind: instance method of Enmap

Param	Type	Description
propOrFn	string | function	The property to test against, or the function to test with
[value]	*	The expected value - only applicable and required if using a property for the first argument

Example

enmap.find('username', 'Bob');

Example

enmap.find(val => val.username === 'Bob');

enmap.exists(prop, value) ⇒ boolean

Searches for the existence of a single item where its specified property's value is identical to the given value (item[prop] === value). Do not use this to check for an item by its ID. Instead, use enmap.has(id). See MDN for details.

Kind: instance method of Enmap

Param	Type	Description
prop	string	The property to test against
value	*	The expected value

Example

if (enmap.exists('username', 'Bob')) {
 console.log('user here!');
}

enmap.filter(fn, [thisArg]) ⇒ Enmap

Identical to Array.filter(), but returns a Enmap instead of an Array.

Kind: instance method of Enmap

Param	Type	Description
fn	function	Function used to test (should return a boolean)
[thisArg]	Object	Value to use as this when executing function

enmap.filterArray(fn, [thisArg]) ⇒ Array

Identical to Array.filter().

Kind: instance method of Enmap

Param	Type	Description
fn	function	Function used to test (should return a boolean)
[thisArg]	Object	Value to use as this when executing function

enmap.map(fn, [thisArg]) ⇒ Array

Identical to Array.map().

Kind: instance method of Enmap

Param	Type	Description
fn	function	Function that produces an element of the new array, taking three arguments
[thisArg]	*	Value to use as this when executing function

enmap.some(fn, [thisArg]) ⇒ boolean

Identical to Array.some().

Kind: instance method of Enmap

Param	Type	Description
fn	function	Function used to test (should return a boolean)
[thisArg]	Object	Value to use as this when executing function

enmap.every(fn, [thisArg]) ⇒ boolean

Identical to Array.every().

Kind: instance method of Enmap

Param	Type	Description
fn	function	Function used to test (should return a boolean)
[thisArg]	Object	Value to use as this when executing function

enmap.reduce(fn, [initialValue]) ⇒ *

Identical to Array.reduce().

Kind: instance method of Enmap

Param	Type	Description
fn	function	Function used to reduce, taking four arguments; accumulator, currentValue, currentKey, and enmap
[initialValue]	*	Starting value for the accumulator

enmap.clone() ⇒ Enmap

Creates an identical shallow copy of this Enmap.

Kind: instance method of Enmap
Example

const newColl = someColl.clone();

enmap.concat(...enmaps) ⇒ Enmap

Combines this Enmap with others into a new Enmap. None of the source Enmaps are modified.

Kind: instance method of Enmap

Param	Type	Description
...enmaps	Enmap	Enmaps to merge

Example

const newColl = someColl.concat(someOtherColl, anotherColl, ohBoyAColl);

enmap.equals(enmap) ⇒ boolean

Checks if this Enmap shares identical key-value pairings with another. This is different to checking for equality using equal-signs, because the Enmaps may be different objects, but contain the same data.

Kind: instance method of Enmap
Returns: boolean - Whether the Enmaps have identical contents

Param	Type	Description
enmap	Enmap	Enmap to compare with

Enmap.multi(names, Provider, options) ⇒ Array.<Map>

Initialize multiple Enmaps easily.

Kind: static method of Enmap
Returns: Array.<Map> - An array of initialized Enmaps.

Param	Type	Description
names	Array.<string>	Array of strings. Each array entry will create a separate enmap with that name.
Provider	EnmapProvider	Valid EnmapProvider object.
options	Object	Options object to pass to the provider. See provider documentation for its options.

Example

// Using local variables and the mongodb provider.
const Enmap = require('enmap');
const Provider = require('enmap-mongo');
const { settings, tags, blacklist } = Enmap.multi(['settings', 'tags', 'blacklist'], Provider, { url: "some connection URL here" });

// Attaching to an existing object (for instance some API's client)
const Enmap = require("enmap");
const Provider = require("enmap-mongo");
Object.assign(client, Enmap.multi(["settings", "tags", "blacklist"], Provider, { url: "some connection URL here" }));