Socket
Socket
Sign inDemoInstall

hyperdrive

Package Overview
Dependencies
Maintainers
5
Versions
271
Alerts
File Explorer

Advanced tools

License
Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

hyperdrive - npm Package Compare versions

Comparing version 11.0.0-alpha.11 to 11.0.0-alpha.12

31

index.js
const Hyperbee = require('hyperbee')
const Hyperblobs = require('hyperblobs')
const isOptions = require('is-options')
const { EventEmitter } = require('events')
const { Writable, Readable } = require('streamx')
const unixPathResolve = require('unix-path-resolve')
const MirrorDrive = require('mirror-drive')
const ReadyResource = require('ready-resource')
module.exports = class Hyperdrive extends EventEmitter {
module.exports = class Hyperdrive extends ReadyResource {
constructor (corestore, key, opts = {}) {

@@ -26,10 +26,7 @@ super()

this.opening = this._open()
this.opening.catch(noop)
this.opened = false
this._openingBlobs = null
this._checkout = _checkout || null
this._batching = !!_files
this._closing = null
this.ready().catch(noop)
}

@@ -65,10 +62,6 @@

update () {
return this.db.feed.update()
update (opts) {
return this.db.feed.update(opts)
}
ready () {
return this.opening
}
checkout (len) {

@@ -96,8 +89,2 @@ return new Hyperdrive(this.corestore, this.key, {

close () {
if (this._closing) return this._closing
this._closing = this._close()
return this._closing
}
async _close () {

@@ -107,3 +94,2 @@ if (this._batching) return this.files.close()

try {
await this.ready()
await this.blobs.core.close()

@@ -113,4 +99,2 @@ await this.db.feed.close()

} catch {}
this.emit('close')
}

@@ -168,5 +152,2 @@

}
this.opened = true
this.emit('ready')
}

@@ -173,0 +154,0 @@

5

package.json
{
"name": "hyperdrive",
"version": "11.0.0-alpha.11",
"version": "11.0.0-alpha.12",
"description": "Hyperdrive is a secure, real-time distributed file system",

@@ -28,2 +28,3 @@ "main": "index.js",

"mirror-drive": "^1.2.0",
"ready-resource": "^1.0.0",
"streamx": "^2.12.4",

@@ -33,3 +34,2 @@ "unix-path-resolve": "^1.0.2"

"devDependencies": {
"@hyperswarm/dht": "^6.2.0",
"@hyperswarm/testnet": "^3.0.0",

@@ -40,2 +40,3 @@ "b4a": "^1.6.0",

"hypercore-crypto": "^3.2.1",
"hyperdht": "^6.5.2",
"hyperswarm": "^4.0.0",

@@ -42,0 +43,0 @@ "random-access-memory": "^6.0.0",

295

README.md

@@ -13,4 +13,2 @@ # Hyperdrive

Note this is the Hyperdrive 11 preview based on Hypercore 10
## Usage

@@ -22,16 +20,32 @@

const corestore = new Corestore('storage')
const store = new Corestore('./storage')
const drive = new Hyperdrive(store)
const drive = new Hyperdrive(corestore, /* optionalKey */)
await drive.put('/blob.txt', Buffer.from('example'))
await drive.put('/images/logo.png', Buffer.from('..'))
await drive.put('/images/old-logo.png', Buffer.from('..'))
const buffer = await drive.get('/blob.txt')
console.log(buffer) // => <Buffer ..> "example"
const entry = await drive.entry('/blob.txt')
console.log(entry) // => { seq, key, value: { executable, linkname, blob, metadata } }
await drive.del('/images/old-logo.png')
await drive.symlink('/images/logo.shortcut', '/images/logo.png')
for await (const file of drive.list('/images')) {
console.log('list', file) // => { key, value }
}
const rs = drive.createReadStream('/blob.txt')
for await (const chunk of rs) {
console.log('rs', chunk) // => <Buffer ..>
}
const ws = drive.createWriteStream('/blob.txt')
ws.write('Hello, ')
ws.write('world!')
ws.write('new example')
ws.end()
ws.on('close', function () {
const rs = drive.createReadStream('/blob.txt')
rs.pipe(process.stdout) // prints Hello, world!
})
ws.once('close', () => console.log('file saved'))
```

@@ -41,28 +55,65 @@

#### `const drive = new Hyperdrive(corestore, [key])`
#### `const drive = new Hyperdrive(store, [key])`
Creates a new Hyperdrive instance.
Creates a new Hyperdrive instance. `store` must be an instance of `Corestore`.
`corestore` must be an instance of `Corestore`.
By default it uses the core at `{ name: 'db' }` from `store`, unless you set the public `key`.
`key` should be a Hypercore public key. If you do not set this, Hyperdrive will use the core at `{ name: 'db' }` in the passed Corestore instance.
#### `await drive.ready()`
Wait for the drive to fully open. In general, you do **NOT** need to wait for `ready` unless checking a synchronous property on `drive` since internals `await` this themselves.
Waits until internal state is loaded.
#### `const buffer = await drive.get(path)`
Use it once before reading synchronous properties like `drive.discoveryKey`, unless you called any of the other APIs.
Returns the blob at `path` in the drive. Internally, Hyperdrive contains a metadata index of entries that "point" to offsets in a `Hyperblobs` instance. Blobs themselves are accessible via `drive.get(path)`, whereas entries are accessible via `drive.entry(path)`. If no blob exists at `path`, returns `null`.
#### `await drive.close()`
#### `const stream = drive.createReadStream(path, options)`
Fully close this drive, including its underlying Hypercore backed datastructures.
Returns a stream that can be used to read out the blob stored in the drive at `path`.
#### `drive.corestore`
`options` are the same as the `options` to `Hyperblobs().createReadStream(path, options)`.
The Corestore instance used as storage.
#### `drive.db`
The underlying Hyperbee backing the drive file structure.
#### `drive.core`
The Hypercore used for `drive.db`.
#### `drive.key`
The public key of the Hypercore backing the drive.
#### `drive.discoveryKey`
The hash of the public key of the Hypercore backing the drive.
Can be used as a `topic` to seed the drive using Hyperswarm.
#### `drive.contentKey`
The public key of the [Hyperblobs](https://github.com/holepunchto/hyperblobs) instance holding blobs associated with entries in the drive.
#### `drive.version`
Number that indicates how many modifications were made, useful as a version identifier.
#### `drive.supportsMetadata`
Boolean indicating if the drive handles or not metadata. Always `true`.
#### `await drive.put(path, buffer, [options])`
Creates a file at `path` in the drive. `options` are the same as in `createWriteStream`.
#### `const buffer = await drive.get(path)`
Returns the blob at `path` in the drive. If no blob exists, returns `null`.
It also returns `null` for symbolic links.
#### `const entry = await drive.entry(path)`
Returns the entry at `path` in the drive. An entry holds metadata about a `path`, currently:
Returns the entry at `path` in the drive. It looks like this:
```js

@@ -73,5 +124,5 @@ {

value: {
executable: Boolean, // whether the blob at path is an executable
linkname: null // if entry not symlink, otherwise a string to the entry this links to
blob: { // a Hyperblob id that can be used to fetch the blob associated with this entry
executable: Boolean, // Whether the blob at path is an executable
linkname: null, // If entry not symlink, otherwise a string to the entry this links to
blob: { // Hyperblobs id that can be used to fetch the blob associated with this entry
blockOffset: Number,

@@ -87,27 +138,39 @@ blockLength: Number,

#### `await drive.del(path)`
Deletes the file at `path` from the drive.
#### `await drive.clear(path)`
Deletes the blob from storage to free up space, but the file structure reference is kept.
#### `await drive.symlink(path, linkname)`
Creates an entry in drive at `path` that points to the entry at `linkname`. Note, if a blob entry currently exists at `path` then `drive.symlink(path, linkname)` will overwrite the entry and `drive.get(path)` will return `null`, while `drive.entry(path)` will return the entry with symlink information.
Creates an entry in drive at `path` that points to the entry at `linkname`.
#### `const hyperblobs = await drive.getBlobs()`
If a blob entry currently exists at `path` then it will get overwritten and `drive.get(key)` will return `null`, while `drive.entry(key)` will return the entry with symlink information.
Returns the hyperblobs instance storing the blobs indexed by drive entries.
#### `const batch = drive.batch()`
```js
const fs = require('fs')
const Corestore = require('corestore')
const Hyperdrive = require('hyperdrive')
Useful for atomically mutate the drive, has the same interface as Hyperdrive.
const drive = new Hyperdrive(new Corestore('storage'))
#### `await batch.flush()`
await drive.put(__filename, fs.readFileSync(__filename))
const bufFromGet = await drive.get(__filename)
Commit a batch of mutations to the underlying drive.
const { value: entry } = await drive.entry(__filename)
const blobs = await drive.getBlobs()
const bufFromEntry = blobs.get(entry.blob)
#### `const stream = drive.list(folder, [options])`
console.log(Buffer.compare(bufFromGet, bufFromEntry)) // prints 0
Returns a stream of all entries in the drive at paths prefixed with `folder`.
`options` include:
```js
{
recursive: true | false // Whether to descend into all subfolders or not
}
```
#### `const stream = drive.readdir(folder)`
Returns a stream of all subpaths of entries in drive stored at paths prefixed by `folder`.
#### `const stream = await drive.entries([options])`

@@ -119,95 +182,65 @@

#### `await drive.put(path, blob, [options])`
#### `const mirror = drive.mirror(out, [options])`
Sets the `blob` in the drive at `path`.
Efficiently mirror this drive into another. Returns a [`MirrorDrive`](https://github.com/holepunchto/mirror-drive#api) instance constructed with `options`.
`path` should be a `utf8` string.
`blob` should be a `Buffer`.
Call `await mirror.done()` to wait for the mirroring to finish.
`options` includes:
#### `const rs = drive.createReadStream(path, [options])`
Returns a stream to read out the blob stored in the drive at `path`.
`options` include:
```js
{
executable: true | false // whether the blob is executable or not
start: Number, // `start` and `end` are inclusive
end: Number,
length: Number, // `length` overrides `end`, they're not meant to be used together
wait: true, // Wait for blocks to be downloaded
timeout: 0 // Wait at max some milliseconds (0 means no timeout)
}
```
#### `ws = drive.createWriteStream(path, [options])`
#### `const ws = drive.createWriteStream(path, [options])`
Stream a blob into the drive at `path`. Options include
Stream a blob into the drive at `path`.
`options` include:
```js
{
executable: true | false // whether the blob is executable or not
executable: Boolean,
metadata: null // Extended file information i.e. arbitrary JSON value
}
```
#### `await drive.del(path)`
#### `await drive.download(folder, [options])`
Removes the `entry` at `path` from the drive. If a blob corresponding to the entry at `path` exists, it is not currently deleted.
Downloads the blobs corresponding to all entries in the drive at paths prefixed with `folder`.
#### `await drive.clear(path)`
`options` are the same as those for `drive.list(folder, [options])`.
Deletes the blob containing the content of the `entry` at `path` from the underlying storage, but leaves the entry itself be (so the entry still exists in the database, but its content does not exist locally anymore). Note that this is a destructive operation which also affects checkouts taken at a time before calling it.
#### `const snapshot = drive.checkout(version)`
#### `const hypercore = drive.core`
Get a read-only snapshot of a previous version.
The underlying Hypercore backing the drive.
#### `const stream = drive.diff(version, folder, [options])`
#### `const buffer = drive.key`
Efficiently create a stream of the shallow changes to `folder` between `version` and `drive.version`.
The public key of the Hypercore backing the drive.
#### `const buffer = drive.discoveryKey`
The hash of the public key of the Hypercore backing the drive, can be used to seed the drive using Hyperswarm.
#### `const buffer = drive.contentKey`
The public key of the Hyperblobs instance holding blobs associated with entries in the drive.
#### `const integer = drive.version`
The version (offset in the underlying Hypercore) of the drive.
#### `const Hyperdrive = drive.checkout(version)`
Checks out a read-only snapshot of a Hyperdrive at a particular version.
Each entry is sorted by key and looks like this:
```js
const fs = require('fs')
const Corestore = require('corestore')
const Hyperdrive = require('hyperdrive')
const drive = new Hyperdrive(new Corestore('storage'))
await drive.put('/fst-file.txt', fs.readFileSync('fst-file.txt'))
const version = drive.version
await drive.put('/snd-file.txt', fs.readFileSync('snd-file.txt'))
const snapshot = drive.checkout(version)
console.log(await drive.get('/snd-file.txt')) // prints Buffer
console.log(await snapshot.get('/snd-file.txt')) // prints null
console.log(Buffer.compare(await drive.get('/fst-file.txt'), await snapshot.get('/fst-file.txt'))) // prints 0
```
#### `const stream = drive.diff(version, folder, [options])`
Efficiently create a stream of the shallow changes to `folder` between `version` and `drive.version`. Each entry is sorted by key and looks like this:
```
{
left: <the entry in folder at drive.version for some path>,
right: <the entry in folder at drive.checkout(version) for some path>
left: Object, // Entry in folder at drive.version for some path
right: Object, // Entry in folder at drive.checkout(version) for some path
}
```
If an entry exists in `drive.version` of the `folder` but not in `version`, then left is set and right will be null, and vice versa.
If an entry exists in `drive.version` of the `folder` but not in `version`, then `left` is set and `right` will be `null`, and vice versa.
#### `await drive.downloadDiff(version, folder, [options])`
Downloads all the blobs in `folder` corresponding to entries in `drive.checkout(version)` that are not in `drive.version`. In other words, downloads all the blobs added to `folder` up to `version` of the drive.
Downloads all the blobs in `folder` corresponding to entries in `drive.checkout(version)` that are not in `drive.version`.
In other words, downloads all the blobs added to `folder` up to `version` of the drive.
#### `await drive.downloadRange(dbRanges, blobRanges)`

@@ -217,45 +250,51 @@

#### `const stream = drive.list(folder, [options])`
#### `const done = drive.corestore.findingPeers()`
Returns a stream of all entries in the drive at paths prefixed with `folder`. Options include:
Indicate to Hyperdrive that you're finding peers in the background, requests will be on hold until this is done.
Call `done()` when your current discovery iteration is done, i.e. after `swarm.flush()` finishes.
#### `const stream = drive.corestore.replicate(isInitiatorOrStream)`
Usage example:
```js
{
recursive: true | false // whether to descend into all subfolders or not
}
const swarm = new Hyperswarm()
const done = drive.corestore.findingPeers()
swarm.on('connection', (socket) => drive.corestore.replicate(socket))
swarm.join(drive.discoveryKey)
swarm.flush().then(done, done)
```
#### `await drive.download(folder, [options])`
See more about how replicate works at [corestore.replicate][store-replicate-docs].
Downloads the blobs corresponding to all entries in the drive at paths prefixed with `folder`. Options are the same as those for `drive.list(folder, [options])`.
#### `const updated = await drive.update([options])`
#### `const stream = drive.readdir(folder)`
Waits for initial proof of the new drive version until all `findingPeers` are done.
Returns a stream of all subpaths of entries in drive stored at paths prefixed by `folder`.
`options` include:
```js
await drive.put('/parent/child', Buffer.from('child'))
await drive.put('/parent/sibling', Buffer.from('sibling'))
for await (const path of drive.readdir('/parent')) console.log(path) // prints "child", then prints "sibling"
{
wait: false
}
```
#### `const mirror = drive.mirror(out, [options])`
Use `drive.corestore.findingPeers()` or `{ wait: true }` to make await `drive.update()` blocking.
Efficiently mirror this drive into another. Returns a [`MirrorDrive`](https://github.com/holepunchto/mirror-drive#api) instance constructed with `options`.
#### `const blobs = await drive.getBlobs()`
Call `await mirror.done()` to wait for the mirroring to finish.
Returns the [Hyperblobs](https://github.com/holepunchto/hyperblobs) instance storing the blobs indexed by drive entries.
#### `const batch = drive.batch()`
```js
await drive.put('/file.txt', Buffer.from('hi'))
Atomically mutate the drive, has the same interface as Hyperdrive.
const buffer1 = await drive.get('/file.txt')
#### `await batch.flush()`
const blobs = await drive.getBlobs()
const entry = await drive.entry('/file.txt')
const buffer2 = await blobs.get(entry.value.blob)
Atomically commit a batch of mutations to the underlying drive.
// => buffer1 and buffer2 are equals
```
#### `await drive.close()`
Close the drive and its underlying Hypercore backed datastructures.
[core-range-docs]: https://github.com/holepunchto/hypercore#const-range--coredownloadrange
[store-replicate-docs]: https://github.com/holepunchto/corestore#const-stream--storereplicateoptsorstream
SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap
  • Changelog

About

Packages

npm

Go

Maven

PyPI

Rubygems

Stay in touch

Get open source security insights delivered straight into your inbox.

  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc