Moneta provides a standard interface for interacting with various kinds of key/value stores. Moneta supports the well-known NoSQL and document based stores.
A short overview of the features:
Moneta::Transformer proxy (Marshal/JSON/YAML and many more)Moneta::Transformer proxy (Zlib, Snappy, LZMA, ...)Moneta::Transformer proxyMoneta::Expires if not supported natively)#increment and #decrement)#create)Moneta::Mutex and Moneta::SemaphoreMoneta::Server) and client (Moneta::Adapters::Client)If you are not yet convinced, you might ask why? What are the goals of the project?
Moneta is tested thoroughly using Travis-CI.
Install Moneta via Rubygems
$ gem install moneta
or add it to your Gemfile
gem 'moneta'
Now you are ready to go:
require 'moneta'
# Create a simple file store
store = Moneta.new(:File, :dir => 'moneta')
# Store some entries
store['key'] = 'value'
# Read entry
store.key?('key') # returns true
store['key'] # returns 'value'
store.close
Out of the box, it supports the following backends. Use the backend name symbol in the Moneta constructor (e.g. Moneta.new(:Memory)).
:Memory):LRUHash):LocalMemCache):Memcached, :MemcachedNative and :MemcachedDalli):DataMapper):ActiveRecord):Sequel):Sqlite):DBM):Cassandra):Daybreak):GDBM):HBase):LevelDB):Redis):Riak):SDBM):KyotoCabinet):TokyoCabinet):TokyoTyrant):TDB):Client works with Moneta::Server):RestClient works with Rack::MonetaRest):Fog):Null)Some of the backends are not exactly based on key/value stores, e.g. the relational ones. These are useful if you already use the corresponding backend in your application. You get a key/value store for free then without installing any additional services and you still have the possibility to upgrade to a real key/value store.
NOTE: The backend matrix is much more readable on rubydoc.info than on github. Go there!
| Adapter | Required gems | Multi-thread safe[1] | Multi-process safe[2] | Atomic increment[8] | Atomic create[9] | Native expires[3] | Persistent | Description |
|---|---|---|---|---|---|---|---|---|
| Persistent stores | ||||||||
| Mongo | mongo | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | MongoDB database |
| Redis | redis | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | Redis database |
| ActiveRecord | activerecord | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ActiveRecord ORM |
| File | - | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | File store |
| Sequel | sequel | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | Sequel ORM |
| TokyoTyrant | tokyotyrant or ruby-tokyotyrant | ✗ | ✓ | ✓ | ✓ | ✗ | ✓ | TokyoTyrant database |
| PStore | - | ✗ | ✓[10] | ✓ | ✓ | ✗ | ✓ | PStore store |
| YAML | - | ✗ | ✓[10] | ✓ | ✓ | ✗ | ✓ | YAML store |
| Sqlite | sqlite3 | ? | ✓[10] | ✓ | ✓ | ✗ | ✓ | Sqlite3 database |
| Daybreak | daybreak | ✗ | (✓)[7] | ✓ | ✓ | ✗ | ✓ | Incredibly fast pure-ruby key/value store Daybreak |
| DBM | - | ✗ | ✗ | ✓ | ✓ | ✗ | ✓ | Berkeley DB using DBM interface or NDBM (Depends on Ruby environment) |
| GDBM | - | ✗ | ✗ | ✓ | ✓ | ✗ | ✓ | GDBM database |
| LevelDB | leveldb | ✗ | ✗ | ✓ | ✓ | ✗ | ✓ | LevelDB database |
| SDBM | - | ✗ | ✗ | ✓ | ✓ | ✗ | ✓ | SDBM database |
| TDB | tdb | ✗ | ✗ | ✓ | ✓ | ✗ | ✓ | TDB database |
| KyotoCabinet | kyotocabinet | ✗ | ✗ | ✓ | ✓ | ✗ | ✓ | KyotoCabinet database |
| TokyoCabinet | tokyocabinet | ✗ | ✗ | ✓ | ✓ | ✗ | ✓ | TokyoCabinet database |
| DataMapper | dm-core, dm-migrations | ✓ | ✓ | ✗ | ✓ | ✗ | ✓ | DataMapper ORM |
| Couch | faraday, multi_json | ✗ | ✓ | ✗ | ✓ | ✗ | ✓ | CouchDB database |
| HBase | hbaserb | ? | ✓ | ✓ | ✗ | ✗ | ✓ | HBase database |
| Cassandra | cassandra | ? | ✓ | ✗ | ✗ | ✓ | ✓ | Cassandra distributed database |
| LocalMemCache | localmemcache | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | LocalMemCache database |
| Fog | fog | ? | ✓ | ✗ | ✗ | ✗ | ✓ | Fog cloud store |
| Riak | riak-client | ✗ | ✓ | ✗ | ✗ | ✗ | ✓ | Riak database |
| Non persistent stores | ||||||||
| MemcachedDalli | dalli | ✓ | ✓ | ✓ | ✓ | ✓ | ✗[4] | Memcached database with Dalli library |
| Memcached | dalli or memcached | ? | ✓ | ✓ | ✓ | ✓ | ✗[4] | Memcached database |
| MemcachedNative | memcached | ✗ | ✓ | ✓ | ✓ | ✓ | ✗[4] | Memcached database with native library |
| Cookie | - | ✗ | (✓)[6] | ✓ | ✓ | ✓ | ✗ | Cookie in memory store |
| LRUHash | - | ✗ | (✓)[6] | ✓ | ✓ | ✗ | ✗ | LRU memory store |
| Memory | - | ✗ | (✓)[6] | ✓ | ✓ | ✗ | ✗ | Memory store |
| Null | - | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | No database |
| Client | - | ✗ | ✓ | ?[5] | ?[5] | ?[5] | ?[5] | Moneta client adapter |
| RestClient | faraday | ✗ | ✓ | ✗ | ✗ | ✗ | ?[5] | Moneta REST client adapter |
Moneta::Lock or by passing the option :threadsafe => true to Moneta#new. There is also Moneta::Pool which can be used to share a store between multiple threads if the store is multi-process safe. I recommend to add the option :threadsafe to ensure thread-safety since for example under JRuby and Rubinius even the basic datastructures are not thread safe due to the lack of a global interpreter lock (GIL). This differs from MRI where some adapters might appear thread safe already but only due to the GIL.Moneta::Shared (See below).Moneta::Expires or by passing the option :expires => true to Moneta#new.Moneta::Semaphore. You can add weak #increment support using the Moneta::WeakIncrement proxy.Moneta::Mutex. You can add weak #create support using the Moneta::WeakCreate proxy.In addition it supports proxies (Similar to Rack middlewares) which add additional features to storage backends:
Moneta::Proxy and Moneta::Wrapper proxy base classesMoneta::Expires to add expiration support to stores which don't support it natively. Add it in the builder using use :Expires.Moneta::Stack to stack multiple stores (Read returns result from first where the key is found, writes go to all stores). Add it in the builder using use(:Stack) {}.Moneta::Transformer transforms keys and values (Marshal, YAML, JSON, Base64, MD5, ...). Add it in the builder using use :Transformer.Moneta::Cache combine two stores, one as backend and one as cache (e.g. Moneta::Adapters::File + Moneta::Adapters::Memory). Add it in the builder using use(:Cache) {}.Moneta::Lock to make store thread safe. Add it in the builder using use :Lock.Moneta::Pool to create a pool of stores as a means of making the store thread safe. Add it in the builder using use(:Pool) {}.Moneta::Logger to log database accesses. Add it in the builder using use :Logger.Moneta::Shared to share a store between multiple processes. Add it in the builder using use(:Shared) {}.Moneta::WeakIncrement and Moneta::WeakCreate to add #create and #increment support without atomicity (weak) to stores which don't support it.Moneta::Transformer)Supported serializers:
:bencode):bert):bson):json):marshal):msgpack):ox):tnet):yaml)Supported value compressors:
:lz4):lzma):lzo):snappy):quicklz):zlib)Special transformers:
:prefix):hmac, useful for Rack::MonetaCookies)The Moneta API is purposely extremely similar to the Hash API with a few minor additions. Every method takes also a optional option hash. In order so support an identical API across stores, Moneta does not support iteration or partial matches.
#initialize(options) options differs per-store, and is used to set up the store.
#[](key) retrieve a key. If the key is not available, return nil.
#load(key, options = {}) retrieve a key. If the key is not available, return nil.
#fetch(key, options = {}, &block) retrieve a key. If the key is not available, execute the
block and return its return value.
#fetch(key, value, options = {}) retrieve a key. If the key is not available, return the value,
#[]=(key, value) set a value for a key. If the key is already used, clobber it.
keys set using []= will never expire.
#store(key, value, options = {}) same as []=, but you can supply options.
#delete(key, options = {}) delete the key from the store and return the current value.
#key?(key, options = {}) true if the key exists, false if it does not.
#increment(key, amount = 1, options = {}) increment numeric value. This is an atomic operation
which is not supported by all stores. Returns current value.
#decrement(key, amount = 1, options = {}) increment numeric value. This is an atomic operation
which is not supported by all stores. Returns current value.
This is just syntactic sugar for incrementing with a negative value.
#create(key, value, options = {}) create entry. This is an atomic operation which is not supported by all stores.
Returns true if the value was created.
#clear(options = {}) clear all keys in this store.
#close close database connection.
#features return array of features, e.g. [:create, :expires, :increment]
#supports?(feature) returns true if store supports a given feature
There is a simple interface to create a store using Moneta.new. You will
get automatic key and value serialization which is provided by Moneta::Transformer.
This allows you to store arbitrary Ruby objects. You can tune some options
when you call Moneta.new. However for very fine tuning use Moneta.build.
store = Moneta.new(:Memcached, :server => 'localhost:11211')
store['key'] = 'value'
store['hash_key'] = {:a => 1, :b => 2}
store['object_key'] = MarshallableRubyObject.new
If you want to have control over the proxies, you have to use Moneta.build:
store = Moneta.build do
# Adds expires proxy
use :Expires
# Transform key using Marshal and Base64 and value using Marshal
use :Transformer, :key => [:marshal, :base64], :value => :marshal
# IMPORTANT: adapter must be defined last for the builder to function properly.
# Memory backend
adapter :Memory
end
You can also directly access the underlying adapters if you don't want to use the Moneta stack.
db = Moneta::Adapters::File.new(:dir => 'directory')
db['key'] = {:a => 1, :b => 2} # This will fail since you can only store Strings
# However for Mongo and Couch this works
# The hash will be mapped directly to a Mongo/Couch document.
db = Moneta::Adapters::Couch.new
db['key'] = {:a => 1, :b => 2}
db = Moneta::Adapters::Mongo.new
db['key'] = {:a => 1, :b => 2}
The Cassandra, Memcached, Redis and Mongo backends support expiration natively.
cache = Moneta::Adapters::Memcached.new
# Or using the builder...
cache = Moneta.build do
adapter :Memcached
end
# Expires in 60 seconds
cache.store(key, value, :expires => 60)
# Never expire
cache.store(key, value, :expires => 0)
cache.store(key, value, :expires => false)
# Update expires time if value is found
cache.load(key, :expires => 30)
cache.key?(key, :expires => 30)
# Or remove the expiration if found
cache.load(key, :expires => false)
cache.key?(key, :expires => 0)
You can add the expires feature to other backends using the Moneta::Expires proxy. But be aware
that expired values are not deleted automatically if they are not looked up.
# Using the :expires option
cache = Moneta.new(:File, :dir => '...', :expires => true)
# or manually by using the proxy...
cache = Moneta::Expires.new(Moneta::Adapters::File.new(:dir => '...'))
# or using the builder...
cache = Moneta.build do
use :Expires
adapter :File, :dir => '...'
end
The stores support the #increment which allows atomic increments of unsigned integer values. If you increment
a non existing value, it will be created. If you increment a non integer value an exception will be raised.
store.increment('counter') # returns 1, counter created
store.increment('counter') # returns 2
store.increment('counter', -1) # returns 1
store.increment('counter', 13) # returns 14
store.increment('counter', 0) # returns 14
store.decrement('counter') # returns 13
store['name'] = 'Moneta'
store.increment('name') # raises an Exception
If you want to access the counter value you have to use raw access to the datastore. This is only important
if you have a Moneta::Transformer somewhere in your proxy stack which transforms the values e.g. with Marshal.
store.increment('counter') # returns 1, counter created
store.load('counter', :raw => true) # returns 1
store.store('counter', '10', :raw => true)
store.increment('counter') # returns 11
Fortunately there is a nicer way to do this using some syntactic sugar!
store.increment('counter') # returns 1, counter created
store.raw['counter'] # returns 1
store.raw.load('counter') # returns 1
store.raw['counter'] = '10'
store.increment('counter') # returns 11
You can also keep the raw store in a variable and use it like this:
counters = store.raw
counters.increment('counter') # returns 1, counter created
counters['counter'] # returns 1
counters.load('counter') # returns 1
counters['counter'] = '10'
counters.increment('counter') # returns 11
The stores support the #create which allows atomic creation of entries. #create returns true
if the value was created.
store.create('key', 'value') # returns true
store.create('key', 'other value') # returns false
Moneta provides shared/distributed synchronization primitives which are shared database-wide between all clients.
Moneta::Mutex allows a single thread to enter a critical section.
mutex = Moneta::Mutex.new(store, 'mutex_key')
mutex.synchronize do
mutex.locked? # returns true
# Synchronized access to counter
store['counter'] += 1
end
begin
mutex.lock
mutex.locked? # returns true
# ...
ensure
mutex.unlock
end
Moneta::Semaphore allows max_concurrent threads to enter a critical section.
semaphore = Moneta::Semaphore.new(store, 'semaphore_counter', max_concurrent)
semaphore.synchronize do
semaphore.locked? # returns true
# ...
end
begin
semaphore.enter
semaphore.locked? # returns true
# ...
ensure
semaphore.leave
end
If an underlying adapter doesn't provide atomic #create or #increment and #decrement you can
use the proxies Moneta::WeakIncrement and Moneta::WeakCreate to add support without atomicity.
But then you have to ensure that the store is not shared by multiple processes and thread-safety is
provided by Moneta::Lock.
For raw data access as described before the class Moneta::OptionMerger is used. It works like this:
# All methods after 'with' get the options passed
store.with(:raw => true).load('key')
# You can also specify the methods
store.with(:raw => true, :only => :load).load('key')
store.with(:raw => true, :except => [:key?, :increment]).load('key')
# Syntactic sugar for raw access
store.raw.load('key')
# Access substore where all keys get a prefix
substore = store.prefix('sub')
substore['key'] = 'value'
store['key'] # returns nil
store['subkey'] # returns 'value'
# Set expiration time for all keys
short_lived_store = store.expires(60)
short_lived_store['key'] = 'value'
You can add proxies to an existing store. This is useful if you want to compress only a few values for example.
compressed_store = store.with(:prefix => 'compressed') do
use :Transformer, :value => :zlib
end
store['key'] = 'this value will not be compressed'
compressed_store['key'] = 'value will be compressed'
Inspired by redis-store there exist integration classes for Rails and Rack/Rack-Cache. You can also use all the Rack middlewares together with Rails and the Sinatra framework. There exist the following integration classes:
Rack::Session::Moneta is a Rack middleware to use Moneta for storing sessionsRack::MonetaStore is a Rack middleware which places a Moneta store in the environment and enables per-request cachingRack::MonetaCookies is a Rack middleware which uses Moneta to store cookiesRack::MonetaRest is a Rack application which exposes a Moneta store via REST/HTTPRack::Cache::Moneta provides meta and entity stores for Rack-CacheActionDispatch::Session::MonetaStore is a Rails middleware to use Moneta for storing sessionsActiveSupport::Cache::MonetaStore is a Rails cache implementation which uses a Moneta store as backendRamaze::Cache::Moneta is integrated into the Ramaze project and allows Ramaze to use
Moneta as caching storeYou can use Moneta as a Rack session store. Use it in your config.ru like this:
require 'rack/session/moneta'
# Use only the adapter name
use Rack::Session::Moneta, :store => :Redis
# Use Moneta.new
use Rack::Session::Moneta, :store => Moneta.new(:Memory, :expires => true)
# Use the Moneta builder
use Rack::Session::Moneta do
use :Expires
adapter :Memory
end
There is a simple middleware which places a Moneta store in the Rack environment at env['rack.moneta_store']. It supports per-request
caching if you add the option :cache => true. Use it in your config.ru like this:
# Add Rack::MonetaStore somewhere in your rack stack
use Rack::MonetaStore, :Memory, :cache => true
run lambda { |env|
env['rack.moneta_store'] # is a Moneta store with per-request caching
}
# Pass it a block like the one passed to Moneta.build
use Rack::MonetaStore do
use :Transformer, :value => :zlib
adapter :Cookie
end
run lambda { |env|
env['rack.moneta_store'] # is a Moneta store without caching
}
If you want to expose your Moneta key/value store via HTTP, you can use the Rack/Moneta REST service. Use it in your config.ru like this:
require 'rack/moneta_rest'
map '/moneta' do
run Rack::MonetaRest.new(:Memory)
end
# Or pass it a block like the one passed to Moneta.build
run Rack::MonetaRest.new do
use :Transformer, :value => :zlib
adapter :Memory
end
You can use Moneta as a Rack-Cache store. Use it in your config.ru like this:
require 'rack/cache/moneta'
use Rack::Cache,
:metastore => 'moneta://Memory?expires=true',
:entitystore => 'moneta://Memory?expires=true'
# Or used named Moneta stores
Rack::Cache::Moneta['named_metastore'] = Moneta.build do
use :Expires
adapter :Memory
end
use Rack::Cache,
:metastore => 'moneta://named_metastore',
:entity_store => 'moneta://named_entitystore'
Use Moneta to store cookies in Rack. It uses the Moneta::Adapters::Cookie. You might
wonder what the purpose of this store or Rack middleware is: It makes it possible
to use all the transformers on the cookies (e.g. :prefix, :marshal and :hmac for value verification).
require 'rack/moneta_cookies'
use Rack::MonetaCookies, :domain => 'example.com', :path => '/path'
run lambda { |env|
req = Rack::Request.new(env)
req.cookies #=> is now a Moneta store!
env['rack.request.cookie_hash'] #=> is now a Moneta store!
req.cookies['key'] #=> retrieves 'key'
req.cookies['key'] = 'value' #=> sets 'key'
req.cookies.delete('key') #=> removes 'key'
[200, {}, []]
}
Add the session store in your application configuration config/environments/*.rb.
require 'moneta'
# Only by adapter name
config.cache_store :moneta_store, :store => :Memory
# Use Moneta.new
config.cache_store :moneta_store, :store => Moneta.new(:Memory)
# Use the Moneta builder
config.cache_store :moneta_store, :store => Moneta.build do
use :Expires
adapter :Memory
end
Add the cache store in your application configuration config/environments/*.rb. Unfortunately the
Moneta cache store doesn't support matchers. If you need these features use a different server-specific implementation.
require 'moneta'
# Only by adapter name
config.cache_store :moneta_store, :store => :Memory
# Use Moneta.new
config.cache_store :moneta_store, :store => Moneta.new(:Memory)
# Use the Moneta builder
config.cache_store :moneta_store, :store => Moneta.build do
use :Expires
adapter :Memory
end
You can use Moneta to build your own key/value server which is shared between
multiple processes. If you run the following code in two different processes,
they will share the same data which will also be persistet in the database shared.db.
require 'moneta'
store = Moneta.build do
use :Transformer, :key => :marshal, :value => :marshal
use :Shared do
use :Cache do
cache do
adapter :LRUHash
end
backend do
adapter :GDBM, :file => 'shared.db'
end
end
end
end
If you want to go further, you might want to take a look at Moneta::Server and Moneta::Adapters::Client which
are used by Moneta::Shared and provide the networking communication. But be aware that they are experimental
and subjected to change. They provide an acceptable performance (for being ruby only), but don't have a stable protocol yet.
You might wonder why I didn't use DRb to implement server and client - in fact my first versions used it, but with much worse performance and it was real fun to implement the networking directly :) There is still much room for improvement and experiments, try EventMachine, try Kgio, ...
If you want something more advanced to handle your objects and relations,
use John Nunemaker's ToyStore which works
together with Moneta. Assuming that Person is a ToyStore::Object you can
add persistence using Moneta as follows:
# Use the Moneta Redis backend
Person.adapter :memory, Moneta.new(:Redis)
Testing is done using Travis-CI. Currently we support Ruby 1.8.7 and 1.9.3.
Benchmarks for each store are done on Travis-CI for each build. Take a look there to compare the speed of the different key value stores for different key/value sizes and size distributions. Feel free to add your own configurations! The impact of Moneta should be minimal since it is only a thin layer on top of the different stores.
Always feel free to open an issue on https://github.com/minad/moneta/issues if something doesn't work as you expect it to work. Feedback is also very welcome!
My only request about patches is that you please try to test them before submitting.
If you want support for another adapter you can at first at it to the list of missing adapters at https://github.com/minad/moneta/issues/16
If you choose to implement an adapter please also add tests. Usually you
only have to add a few lines to script/generate-specs to generate appropriate
tests for your adapter. Please check also if travis.yml needs changes, for example
if you need to start additional services.
Check if the default settings in Moneta#new are appropriate for your adapter. If not specify a better one.
Don't forget to edit the README.md and the CHANGES.
FAQs
Unknown package
We found that moneta demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 4 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.
Research
Security News
A threat actor's playbook for exploiting the npm ecosystem was exposed on the dark web, detailing how to build a blockchain-powered botnet.
Security News
NVD’s backlog surpasses 20,000 CVEs as analysis slows and NIST announces new system updates to address ongoing delays.