level-js
An abstract-leveldown
compliant store on top of IndexedDB, which is in turn implemented on top of LevelDB which brings this whole shebang full circle.
Table of Contents
Click to expand
Background
Here are the goals of level-js
:
- Store large amounts of data in modern browsers
- Pass the full
abstract-leveldown
test suite - Support
Buffer
keys and values - Support all key types of IndexedDB Second Edition
- Support all value types of the structured clone algorithm except for
null
and undefined
- Be as fast as possible
- Sync with multilevel over ASCII or binary transports.
Being abstract-leveldown
compliant means you can use many of the Level modules on top of this library. For some demos of it working, see @brycebaril's presentation Path of the NodeBases Jedi.
Example
If you are upgrading: please see UPGRADING.md.
var levelup = require('levelup')
var leveljs = require('level-js')
var db = levelup(leveljs('bigdata'))
db.put('hello', Buffer.from('world'), function (err) {
if (err) throw err
db.get('hello', function (err, value) {
if (err) throw err
console.log(value.toString())
})
})
In ES6 browsers:
const levelup = require('levelup')
const leveljs = require('level-js')
const db = levelup(leveljs('bigdata'))
await db.put('hello', Buffer.from('world'))
const value = await db.get('hello')
Browser Support
Type Support
Unlike leveldown
, level-js
does not stringify keys or values. This means that in addition to strings and Buffers you can store almost any JavaScript type without the need for encoding-down
.
Values
All value types of the structured clone algorithm are supported except for null
and undefined
. Depending on the environment, this includes:
- Number, including
NaN
, Infinity
and -Infinity
- String, Boolean, Date, RegExp, Array, Object
- ArrayBuffer or a view thereof (typed arrays);
- Map, Set, Blob, File, FileList, ImageData (limited support).
In addition level-js
stores Buffer
values without transformation. This works in all target environments because Buffer
is a subclass of Uint8Array
, meaning such values can be passed to IndexedDB
as-is.
When getting or iterating binary values, regardless of whether they were stored as a Buffer
, ArrayBuffer
or a view thereof, values will return as a Buffer
. This behavior can be disabled, in which case ArrayBuffer
returns as ArrayBuffer
, typed arrays return as typed arrays and Buffer
returns as Uint8Array
:
db.get('key', { asBuffer: false })
db.iterator({ valueAsBuffer: false })
If the environment does not support a type, it will throw an error which level-js
catches and passes to the callbacks of put
or batch
. For example, IE does not support typed array values. At the time of writing, Chrome is the only browser that supports all types listed above.
Due to the special meaning that null
and undefined
have in abstract-leveldown
iterators and Node.js streams, values of this type are converted to empty strings prior to storage.
Keys
All key types of IndexedDB Second Edition are supported. Depending on the environment, this includes:
- Number, including
Infinity
and -Infinity
, but not NaN
- Date, except invalid (
NaN
) - String
- ArrayBuffer or a view thereof (typed arrays);
- Array, except cyclical, empty and sparse arrays. Elements must be valid types themselves.
In addition you can use Buffer
keys, giving level-js
the same power as implementations like leveldown
and memdown
. When iterating binary keys, regardless of whether they were stored as Buffer
, ArrayBuffer
or a view thereof, keys will return as a Buffer
. This behavior can be disabled, in which case binary keys will always return as ArrayBuffer
:
db.iterator({ keyAsBuffer: false })
Note that this behavior is slightly different from values due to the way that IndexedDB works. IndexedDB stores binary values using the structured clone algorithm, which preserves views, but it stores binary keys as an array of octets, so that it is able to compare and sort differently typed keys.
If the environment does not support a type, it will throw an error which level-js
catches and passes to the callbacks of get
, put
, del
, batch
or an iterator. Exceptions are:
null
and undefined
: rejected early by abstract-leveldown
- Boolean and
NaN
: though invalid per the IndexedDB specification, they are converted to strings for abstract-leveldown
compatibility; - Binary and array keys: if not supported by the environment,
level-js
falls back to String(key)
.
Normalization
If you desire normalization for keys and values (e.g. to stringify numbers), wrap level-js
with encoding-down
. Alternatively install level-browserify
which conveniently bundles levelup
, level-js
and encoding-down
. Such an approach is also recommended if you want to achieve universal (isomorphic) behavior or to smooth over type differences between browsers. For example, you could have leveldown
in a backend and level-js
in the frontend.
Another reason you might want to use encoding-down
is that the structured clone algorithm, while rich in types, can be slower than JSON.stringify
.
Buffer vs ArrayBuffer
For interoperability it is recommended to use Buffer
as your binary type. While we recognize that Node.js core modules are moving towards supporting ArrayBuffer
and views thereof, Buffer
remains the primary binary type in the Level ecosystem.
That said: if you want to put()
an ArrayBuffer
you can! Just know that it will come back as a Buffer
by default. If you want to get()
or iterate stored ArrayBuffer
data as an ArrayBuffer
, you have a few options. Without encoding-down
:
const db = levelup(leveljs('mydb'))
const value1 = await db.get('key', { asBuffer: false })
const value2 = await db.get('key')
const value3 = value2.buffer
With encoding-down
(or level-browserify
) you can use the id
encoding to selectively bypass encodings:
const encode = require('encoding-down')
const db = levelup(encode(leveljs('mydb'), { valueEncoding: 'binary' }))
const value1 = await db.get('key', { valueEncoding: 'id' })
const value2 = await db.get('key')
const value3 = value2.buffer
Install
With npm do:
npm install level-js
Not to be confused with leveljs.
This library is best used with browserify.
API
db = leveljs(location[, options])
Returns a new leveljs
instance. location
is the string name of the IDBDatabase
to be opened, as well as the object store within that database. The database name will be prefixed with options.prefix
.
options
The optional options
argument may contain:
prefix
(string, default: 'level-js-'
): Prefix for IDBDatabase
name.version
(string | number, default: 1
): The version to open the database with.
See IDBFactory#open
for more details.
Running Tests
git clone git@github.com:Level/level-js.git
cd level-js
npm install
npm test
It will print out a URL to open in a browser of choice.
Big Thanks
Cross-browser Testing Platform and Open Source ♥ Provided by Sauce Labs.
License
MIT © 2012-present Max Ogden and Contributors.