savior

Savior Database

Embedded schema-less database library backed by LMDB.

Currently in development and not stable yet. The interface and disk format may change in future versions without notice. You probably shouldn't use it to store important data.

Permissively licensed with the ISC License.

Goals

• provide a language-neutral disk format
• utilize LMDB's properties like transactions, append-only writes, and zero-copy reads
• provide intuitive schema-less entity modeling and querying interface
• store an immutable history of all values in the database

Developer Interface

There are a few actions you can perform on entities in a transaction:

• store
• fetch
• update
• query
• delete

An entity is conceptually like a dictionary with attributes and values.

A table is an isolated section of the database for storing entities, usually with a similar set of attributes.

Disk Format

All entities of the same type are stored in one inner LMDB database There is no guarantee that entities have a consistent schema.

Entities are stored on disk as a series of timestamped attribute changes.

• key-value entries have the form (uuid, timestamp, attribute) -> value
• uuid is an auto-generated entity ID
• timestamp is an auto-generated timestamp of when the entry was appended
• attribute and value are the updated entity key value entries

Data is not modified for updates, only appended to indicate the entity changed.

The storage format version is stored as an integer in a special inner database named _metadata, at a key named version. The _metadata table's keys and values are also encoded with MessagePack.

You can access the storage format version via Database.VERSION.

Changelog

This document follows the Keep a Changelog format.

Releases are available on PyPI as savior.

0.2.3 - 2019-09-06

Added

• added Transaction.delete method to permanently remove an entity

0.2.2 - 2019-08-06

Fixed

• fixed bug where created_at dates were all from the first entity

0.2.1 - 2019-08-06

Changed

• opening a database adds a schema version identifier integer to databases that don't have one

Added

• added Database.VERSION for getting storage format version
• added created_at option to query to add an attribute to entities indicating when they were created

0.2.0 - 2019-07-15

Removed

• removed Database.open_tables since tables are now required when opening a database

Changed

• renamed Transaction.create to Transaction.store for consistency with Transaction.fetch
• changed Database.open behavior to require table names on initialization
• changed Transaction methods to raise error if a table name is used that was not used to open the database with

0.1.2 - 2019-07-15

Added

• Database.create_tables to create tables in database
• Database.open_tables to open existing tables in database
• Database.close to close database manually

0.1.1 - 2019-07-15

Added

• options for manually closing transactions for when it's not possible to use context managers

0.1.0 - 2019-07-15

Added

• initial schema-less immutable interface over LMDB