Security News
ESLint is Now Language-Agnostic: Linting JSON, Markdown, and Beyond
ESLint has added JSON and Markdown linting support with new officially-supported plugins, expanding its versatility beyond JavaScript.
interface-datastore
Advanced tools
Implementation of the datastore interface in JavaScript
has(key, [options])
-> Promise<Boolean>
put(key, value, [options])
-> Promise
putMany(source, [options])
-> AsyncIterator<{ key: Key, value: Buffer }>
get(key, [options])
-> Promise<Buffer>
getMany(source, [options])
-> AsyncIterator<Buffer>
delete(key, [options])
-> Promise
deleteMany(source, [options])
-> AsyncIterator<Key>
query(query, [options])
-> AsyncIterable<Buffer>
batch()
open()
-> Promise
close()
-> Promise
src/memory
datastore-level
(supports any levelup compatible backend)datstore-fs
datastore-core/src/mount
datstore-core/src/keytransform
datastore-core/src/sharding
datstore-core/src/tiered
datastore-core/src/namespace
If you want the same functionality as go-ds-flatfs, use sharding with fs.
const FsStore = require('datastore-fs')
const ShardingStore = require('datastore-core').ShardingDatatstore
const NextToLast = require('datastore-core').shard.NextToLast
const fs = new FsStore('path/to/store')
// flatfs now works like go-flatfs
const flatfs = await ShardingStore.createOrOpen(fs, new NextToLast(2))
An adapter is made available to make implementing your own datastore easier:
const { Adapter } = require('interface-datastore')
class MyDatastore extends Adapter {
constructor () {
super()
}
async put (key, val) {
// your implementation here
}
async get (key) {
// your implementation here
}
// etc...
}
See the MemoryDatastore for an example of how it is used.
$ npm install interface-datastore
The type definitions for this package are available on http://definitelytyped.org/. To install just use:
$ npm install -D @types/interface-datastore
const MemoryStore = require('interface-datastore').MemoryDatastore
const MountStore = require('datastore-core').MountDatastore
const Key = require('interface-datastore').Key
const store = new MountStore({ prefix: new Key('/a'), datastore: new MemoryStore() })
Available under src/tests.js
describe('mystore', () => {
require('interface-datastore/src/tests')({
async setup () {
return instanceOfMyStore
},
async teardown () {
// cleanup resources
}
})
})
### Aborting requests
Most API methods accept an AbortSignal as part of an options object. Implementations may listen for an abort
event emitted by this object, or test the signal.aborted
property. When received implementations should tear down any long-lived requests or resources created.
### Concurrency
The streaming (put|get|delete)Many
methods are intended to be used with modules such as it-parallel-batch to allow calling code to control levels of parallelisation. The batching method ensures results are returned in the correct order, but interface implementations should be thread safe.
const batch = require('it-parallel-batch')
const source = [{
key: ..,
value: ..
}]
// put values into the datastore concurrently, max 10 at a time
for await (const { key, data } of batch(store.putMany(source), 10)) {
console.info(`Put ${key}`)
}
To allow a better abstraction on how to address values, there is a Key
class which is used as identifier. It's easy to create a key from a Buffer
or a string
.
const a = new Key('a')
const b = new Key(Buffer.from('hello'))
The key scheme is inspired by file systems and Google App Engine key model. Keys are meant to be unique across a system. They are typically hierarchical, incorporating more and more specific namespaces. Thus keys can be deemed 'children' or 'ancestors' of other keys:
new Key('/Comedy')
new Key('/Comedy/MontyPython')
Also, every namespace can be parameterized to embed relevant object information. For example, the Key name
(most specific namespace) could include the object type:
new Key('/Comedy/MontyPython/Actor:JohnCleese')
new Key('/Comedy/MontyPython/Sketch:CheeseShop')
new Key('/Comedy/MontyPython/Sketch:CheeseShop/Character:Mousebender')
Implementations of this interface should make the following methods available:
has(key, [options])
-> Promise<Boolean>
Check for the existence of a given key
Name | Type | Description |
---|---|---|
key | Key | The key to check the existance of |
options | Object | An options object, all properties are optional |
options.signal | AbortSignal | A way to signal that the caller is no longer interested in the outcome of this operation |
const exists = await store.has(new Key('awesome'))
if (exists) {
console.log('it is there')
} else {
console.log('it is not there')
}
put(key, value, [options])
-> Promise
Store a value with the given key.
Name | Type | Description |
---|---|---|
key | Key | The key to store the value under |
value | Buffer | Value to store |
options | Object | An options object, all properties are optional |
options.signal | AbortSignal | A way to signal that the caller is no longer interested in the outcome of this operation |
await store.put([{ key: new Key('awesome'), value: Buffer.from('datastores') }])
console.log('put content')
putMany(source, [options])
-> AsyncIterator<{ key: Key, value: Buffer }>
Store many key-value pairs.
Name | Type | Description |
---|---|---|
source | AsyncIterator<{ key: Key, value: Buffer }> | The key to store the value under |
value | Buffer | Value to store |
options | Object | An options object, all properties are optional |
options.signal | AbortSignal | A way to signal that the caller is no longer interested in the outcome of this operation |
const source = [{ key: new Key('awesome'), value: Buffer.from('datastores') }]
for await (const { key, value } of store.putMany(source)) {
console.info(`put content for key ${key}`)
}
get(key, [options])
-> Promise<Buffer>
Name | Type | Description |
---|---|---|
key | Key | The key retrieve the value for |
options | Object | An options object, all properties are optional |
options.signal | AbortSignal | A way to signal that the caller is no longer interested in the outcome of this operation |
Retrieve the value stored under the given key.
const value = await store.get(new Key('awesome'))
console.log('got content: %s', value.toString('utf8'))
// => got content: datastore
getMany(source, [options])
-> AsyncIterator<Buffer>
Name | Type | Description |
---|---|---|
source | AsyncIterator<Key> | One or more keys to retrieve values for |
options | Object | An options object, all properties are optional |
options.signal | AbortSignal | A way to signal that the caller is no longer interested in the outcome of this operation |
Retrieve a stream of values stored under the given keys.
for await (const value of store.getMany([new Key('awesome')])) {
console.log('got content: %s', value.toString('utf8'))
// => got content: datastore
}
delete(key, [options])
-> Promise
Delete the content stored under the given key.
Name | Type | Description |
---|---|---|
key | Key | The key to remove the value for |
options | Object | An options object, all properties are optional |
options.signal | AbortSignal | A way to signal that the caller is no longer interested in the outcome of this operation |
await store.delete(new Key('awesome'))
console.log('deleted awesome content :(')
deleteMany(source, [options])
-> AsyncIterator<Key>
Delete the content stored under the given keys.
Name | Type | Description |
---|---|---|
source | AsyncIterator<Key> | One or more keys to remove values for |
options | Object | An options object, all properties are optional |
options.signal | AbortSignal | A way to signal that the caller is no longer interested in the outcome of this operation |
const source = [new Key('awesome')]
for await (const key of store.deleteMany(source)) {
console.log(`deleted content with key ${key}`)
}
query(query, [options])
-> AsyncIterable<Buffer>
Search the store for some values. Returns an AsyncIterable with each item being a Buffer.
Name | Type | Description |
---|---|---|
query | Object | A query object, all properties are optional |
query.prefix | String | Only return values where the key starts with this prefix |
query.filters | Array<Function(Buffer) -> Boolean> | Filter the results according to the these functions |
query.orders | Array<Function(Array<Buffer>) -> Array<Buffer>> | Order the results according to these functions |
query.limit | Number | Only return this many records |
query.offset | Number | Skip this many records at the beginning |
options | Object | An options object, all properties are optional |
options.signal | AbortSignal | A way to signal that the caller is no longer interested in the outcome of this operation |
// retrieve __all__ values from the store
let list = []
for await (const value of store.query({})) {
list.push(value)
}
console.log('ALL THE VALUES', list)
batch()
This will return an object with which you can chain multiple operations together, with them only being executed on calling commit
.
const b = store.batch()
for (let i = 0; i < 100; i++) {
b.put(new Key(`hello${i}`), Buffer.from(`hello world ${i}`))
}
await b.commit()
console.log('put 100 values')
put(key, value)
Queue a put operation to the store.
Name | Type | Description |
---|---|---|
key | Key | The key to store the value under |
value | Buffer | Value to store |
delete(key)
Queue a delete operation to the store.
Name | Type | Description |
---|---|---|
key | Key | The key to remove the value for |
commit([options])
-> Promise<void>
Write all queued operations to the underyling store. The batch object should not be used after calling this.
Name | Type | Description |
---|---|---|
options | Object | An options object, all properties are optional |
options.signal | AbortSignal | A way to signal that the caller is no longer interested in the outcome of this operation |
const batch = store.batch()
batch.put(new Key('to-put'), Buffer.from('hello world'))
batch.del(new Key('to-remove'))
await batch.commit()
open()
-> Promise
Opens the datastore, this is only needed if the store was closed before, otherwise this is taken care of by the constructor.
close()
-> Promise
Close the datastore, this should always be called to ensure resources are cleaned up.
PRs accepted.
Small note: If editing the Readme, please conform to the standard-readme specification.
MIT 2017 © IPFS
FAQs
datastore interface
We found that interface-datastore demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
ESLint has added JSON and Markdown linting support with new officially-supported plugins, expanding its versatility beyond JavaScript.
Security News
Members Hub is conducting large-scale campaigns to artificially boost Discord server metrics, undermining community trust and platform integrity.
Security News
NIST has failed to meet its self-imposed deadline of clearing the NVD's backlog by the end of the fiscal year. Meanwhile, CVE's awaiting analysis have increased by 33% since June.