node-persist

node-persist

(localStorage on the server)

Super-easy (and fast) persistent data structures in Node.js, modeled after HTML5 localStorage

Node-persist doesn't use a database. Instead, JSON documents are stored in the file system for persistence. Because there is no network overhead and your data is just in-memory, node-persist is just about as fast as a database can get. Node-persist uses the HTML5 localStorage API, so it's easy to learn.

This is still a work in progress. Send pull requests please.

Install

$ npm install node-persist

Then in code you can do:

var storage = require('node-persist');

Basic Example

Async example

//you must first call storage.init

storage.init( /* options ... */ ).then(function() {
  //then start using it
  storage.setItem('name','yourname')
  .then(function() {

    return storage.getItem('name')
  })
  .then(function(value) {

    console.log(value); // yourname
  })
});

Sync example

//you must first call storage.initSync
storage.initSync();

//then start using it
storage.setItemSync('name','yourname');
console.log(storage.getItemSync('name')); // yourname

Run the examples:

$ cd examples/examplename
$ node examplename.js
$ open up localhost:8080

2.0.0 change logs

Non-backward changes

• filenames on the file system are now md5 hashed now and the structure of the saved data has changed to include the ttl in them.
• no longer need/support a options.ttlDir, since the ttls are now stored in the same file as each value
• added expiredInterval option
• added forgiveParseErrors option

1.0.0 change logs

Mostly non-backward changes

• storage.getItem() now returns a promise
• storage.valuesWithKeyMatch() no longer accepts a callback
• storage.values() no longer accepts a callback
• storage.key() is gone
• The default dir is now process.cwd() + (dir || '.node-persist/storage'), unless you use an absolute path
• added storage.get(), alias to getItem()
• added storage.set(), alias to setItem()
• added storage.del(), storage.rm(), as aliases to removeItem()
• Keys, on the file system are base64 encoded with the replacement of the /

API Documentation

init(options, [callback]) - asynchronous*, returns Promise

This function reads what's on disk and loads it into memory, if the storage dir is new, it will create it

Options

You can pass init() or initSync() an options object to customize the behavior of node-persist

These are the defaults

storage.init({
	dir: 'relative/path/to/persist',

	stringify: JSON.stringify,

	parse: JSON.parse,

	encoding: 'utf8',

	logging: false,  // can also be custom logging function

	continuous: true, // continously persist to disk

	interval: false, // milliseconds, persist to disk on an interval

	ttl: false, // ttl* [NEW], can be true for 24h default or a number in MILLISECONDS

	expiredInterval: 2 * 60 * 1000, // [NEW] every 2 minutes the process will clean-up the expired cache

    // in some cases, you (or some other service) might add non-valid storage files to your
    // storage dir, i.e. Google Drive, make this true if you'd like to ignore these files and not throw an error
    forgiveParseErrors: false // [NEW]

}, /* optional callback */ ).then(onSuccess, onError); // or use the promise

* With ttl (time to live), it is recommended that you use getItem(key, callback) or getItemSync(key) since, if a ttl of a certain key is expired the key-file is immediately deleted from disk, the callback will execute whenever that happends, if there is no ttl used or it has expired yet, the callback will also immediately execute in a synchronous fashion.

Node-persist has 3 ways of running:

1. By default, keys will be persisted after every call of setItem
2. If you set an interval, node-persist will persist changed keys at that interval instead of after every call of setItem.
3. If you set continuous to false and don't specify an interval, keys aren't persisted automatically, giving you complete control over when to persist them.

initSync(options) - synchronous, throws Error on failure

like init() but synchronous,

getItem(key, [callback]) - returns promise,

This function will get a key from your database in memory

// callback
storage.getItem('name', function (err, value) {
    // use value here after making sure expired-ttl key deletion has occured, in that case value === undefined
});

// promise
storage.getItem('obj').then(function(value) {

})

getItemSync(key) - returns value

All synchronous part along with the deletion of an expired-ttl key, if options.ttl is used

setItem(key, value, [options, callback]) - asynchronous*, returns Promise

This function sets 'key' in your database to 'value'. It also sets a flag, notifying that 'key' has been changed and needs to be persisted in the next sweep. Because the flag must be set for the object to be persisted, it is best to use node-persist in a functional way, as shown below.

storage.setItem('fibonacci',[0,1,1,2,3,5,8]);
storage.setItem(42,'the answer to life, the universe, and everything.', function(err) {
    // done
});
storage.setItem(42,'the answer to life, the universe, and everything.', {ttl: 1000*60 /* 1 min */ }, function(err) {
    // done
});

var batman = storage.getItem('batman');
batman.sidekick = 'Robin';

// using the promise
storage.setItem('batman', batman).then(
  function() {
    // success
  },
  function() {
     // error
  })

* The only option available when calling setItem(key, value, option) is {ttl: $milliseconds}

* setItem() is asynchronous, however, depending on your global options, the item might not persist to disk immediately, in the case where you set options.interval or options.continuous=false, your (optional) callback or your returned promise from this function will still get resolved immediately, even if the value has not been persisted to disk yet, which could be either waiting for the interval to kick in or for your manual call to persist() - kind of how the redis database works.

setItemSync(key, value, [options]) - synchronous, throws Error on failure

If you want to immediately persist to disk, regardless of the this.options.interval and this.options.continuous settings, use this function. The only option available when calling setItemSync(key, value, option) is {ttl: $milliseconds}

storage.setItemSync('foo', 'bar');
storage.setItemSync('hello', 'world', {ttl: 1000 * 60 /* ttl 1 minute */})

removeItem(key, [callback]) - asynchronous, returns Promise

This function removes key in the database if it is present, and immediately deletes it from the file system asynchronously. If ttl is used, the corrresponding ttl-key is removed as well

storage.removeItem('me', /* optional callback */ function(err) {
  // done 
}).then(onSuccess, onError); // or use the promise

removeItemSync(key) - synchronous, throws Error on failure

just like removeItem, but synchronous

storage.removeItemSync('me');

clear([callback]) - asynchronous, returns Promise

This function removes all keys in the database, and immediately deletes all keys from the file system asynchronously.

clearSync() - synchronous, throws Error on failure

like clear() but synchronous

values() - synchronous, returns array

This function returns all of the values in the database in memory.

storage.setItem("batman", {name: "Bruce Wayne"});
storage.setItem("superman", {name: "Clark Kent"});
console.log(storage.values()); //output: [{name: "Bruce Wayne"},{name: "Clark Kent"}]

values() - returns array

var values = storage.values();

valuesWithKeyMatch(match) - synchronous, returns array

This function returns all of the values in the database matching a string or RegExp

storage.setItem("batman", {name: "Bruce Wayne"});
storage.setItem("superman", {name: "Clark Kent"});
storage.setItem("hulk", {name: "Bruce Banner"});
console.log(storage.valuesWithKeyMatch('man')); //output: [{name: "Bruce Wayne"},{name: "Clark Kent"}]
// also accepts a Regular Expression
console.log(storage.valuesWithKeyMatch(/man/)); //output: [{name: "Bruce Wayne"},{name: "Clark Kent"}]

valuesWithKeyMatch(match) - synchronous, returns array

var values = storage.valuesWithKeyMatch('man');

keys() - synchronous, returns array

this function returns an array of all the keys in the database. This function returns the number of keys stored in the database.

length() - synchronous, returns number

This function returns the number of keys stored in the database.

forEach(callback) - synchronous, assuming callback is as well.

This function iterates over each key/value pair and executes a callback.

storage.forEach(function(key, value) {
	// use key and value
});

Fine-grained control

Make sure you set continuous:false in the options hash, and you don't set an interval

persist([callback]) - asynchronous, returns Promise

These function can be used to manually persist the database

storage.persist( /* optional callback */ function(err) {
    // when done
}).then(onSuccess, onError); // or you can use the promise

persistSync() - synchronous, throws Error on failure

like persist() but synchronous

storage.persistSync();

note:

Both persist(), persistSync(), persistKey(), and persistKeySync() will automatically persist the ttl keys/values in the persistance process

persistKey(key, [callback]) - asynchronous, returns Promise

This function manually persist a 'key' within the database

storage.setItem('name','myname');
storage.persistKey('name', /* optional callback */ function(err) {
    // when done
}).then(onSuccess, onError); // or you can use the promise

persistKeySync(key)

like persistKey() but synchronous

storage.setItem('name','myname');
storage.persistKeySync('name');

Factory method

create(options) - synchronous, static method

If you choose to create multiple instances of storage, you can. Just avoid using the same dir for the storage location. You still have to call init or initSync after create - you can pass your configs to either create or init/Sync

The reason we don't call init in the constructor (or when you create) because we can only do so for the initSync version, the async init returns a promise, and in order to maintain that API, we cannot return the promise in the constructor, so init must be called on the instance of new LocalStorage();

var storage = require('node-persist');
var myStorage = storage.create({dir: 'myDir', ttl: 3000});
myStorage.init().then(function() { // or you can use initSync()
   // ...
});

Contributing

Tests

npm install
npm test

Simon Last