Hyperdrive

Hyperdrive stores filesystem metadata in a Hyperbee and file contents in a blob store, typically backed by one Corestore. Use this page for the instance surface; use the linked how-to guides for end-to-end replication flows.

Install

npm i hyperdrive

Quickstart

import Corestore from 'corestore'
import Hyperdrive from 'hyperdrive'

const store = new Corestore('./storage')
const drive = new Hyperdrive(store)

await drive.ready()

await drive.put('/hello.txt', Buffer.from('hello world'), {
  metadata: { type: 'text/plain' }
})

const buffer = await drive.get('/hello.txt')
const entry = await drive.entry('/hello.txt')

console.log(buffer.toString())
console.log(entry?.value.metadata)

await drive.close()

API Reference

Constructor and lifecycle

new Hyperdrive(store, [key])

src

Creates a new Hyperdrive instance. store must be an instance of Corestore.

const Corestore = require('corestore')

const store = new Corestore('./storage')
const drive = new Hyperdrive(store)

await drive.ready()

Waits until internal state is loaded.

await drive.close()

Fully close this drive, including its underlying Hypercore backed datastructures.

await drive.purge()

src

Purge both cores (db and blobs) from your storage, completely removing all the drive's data.

• Returns: Promise<void> — Resolves once both cores are purged from storage.
• Throws: if called on a non-main session — only the main session can be purged.

await drive.purge()

Drive properties

drive.corestore

src

The Corestore instance used as storage.

• Returns: Corestore

drive.db

src

The underlying Hyperbee backing the drive file structure.

• Returns: Hyperbee

drive.core

src

The Hypercore used for drive.db.

• Returns: Hypercore

drive.id

src

String containing the id (z-base-32 of the public key) identifying this drive.

• Returns: string

drive.key

src

The public key of the Hypercore backing the drive.

• Returns: Buffer

drive.discoveryKey

src

The hash of the public key of the Hypercore backing the drive.

• Returns: Buffer

drive.contentKey

src

The public key of the Hyperblobs instance holding blobs associated with entries in the drive.

• Returns: Buffer

drive.writable

src

Boolean indicating if we can write or delete data in this drive.

• Returns: boolean

drive.readable

src

Boolean indicating if we can read from this drive. After closing the drive this will be false.

• Returns: boolean

drive.version

src

Number that indicates how many modifications were made, useful as a version identifier.

• Returns: number

drive.supportsMetadata

src

Boolean indicating if the drive handles or not metadata. Always true.

• Returns: boolean

Files

await drive.put(path, buffer, [options])

src

Creates a file at path in the drive. options are the same as in createWriteStream.

• Returns: Promise<void> — Resolves once the file has been written.

await drive.put('/blob.txt', Buffer.from('example'))

await drive.get(path, [options])

src

Reads the blob stored at path and resolves with its contents as a Buffer. Resolves with null when no blob exists at path, and also returns null for symbolic links.

• Returns: Promise<Buffer\|null> — The blob at path, or null if the path does not exist or is a symbolic link.
• Throws: BLOCK_NOT_AVAILABLE if a required block is not available.

await drive.entry(path, [options])

src

Resolves with the entry stored at path, or null if no entry exists.

• Returns: Promise<object> — the entry at path in the drive.
• Throws: if a symlink chain is recursive or exceeds 16 hops when follow is enabled.

await drive.exists(path)

src

Checks whether an entry exists at path.

• Returns: Promise<boolean> — true if the entry at path does exists, otherwise false.

const exists = await drive.exists('/blob.txt')

await drive.has(path)

src

Checks if path is saved to local store already.

• Returns: Promise<boolean> — true if the blob (or every blob under a folder) is stored locally.

const exists = await drive.has('/blob.txt')

await drive.del(path)

src

Deletes the file at path from the drive.

• Returns: Promise<void> — Resolves once the entry has been deleted.

await drive.del('/blob.txt')

await drive.clear(path, [options])

src

Deletes the blob from storage to free up space, but the file structure reference is kept.

• Returns: Promise<object> — A { blocks } object when diff is enabled, otherwise null.

await drive.clearAll([options])

src

Deletes all the blobs from storage to free up space, similar to how drive.clear() works.

• Returns: Promise<object> — A { blocks } object when diff is enabled, otherwise null.

await drive.symlink(path, linkname)

src

Creates an entry in drive at path that points to the entry at linkname.

• Returns: Promise<void> — Resolves once the symlink entry has been created.

await drive.symlink('/images/logo.shortcut', '/images/logo.png')

drive.createReadStream(path, [options])

src

Returns a readable stream of the blob stored in the drive at path.

• Returns: Readable — a stream to read out the blob stored in the drive at path.

drive.createWriteStream(path, [options])

src

Stream a blob into the drive at path.

• Returns: Writable — A writable stream; data written is stored as the file's blob.

Directories and listings

drive.list(folder, [options])

src

Returns a stream of all entries in the drive at paths prefixed with folder.

• Returns: Readable — a stream of all entries in the drive at paths prefixed with folder.

drive.readdir(folder, [options])

src

Returns a stream of the immediate subpaths of entries stored at paths prefixed by folder.

• Returns: Readable — a stream of all subpaths of entries in drive stored at paths prefixed by folder.

await drive.entries([range], [options])

src

Returns a read stream of the drive entries within the given Hyperbee range.

• Returns: Readable — a read stream of entries in the drive.

for await (const entry of drive.entries()) console.log(entry.key)

drive.compare(entryA, entryB)

src

Compares two entries by their sequence number.

• Returns: number — 0 if entries are the same, 1 if entryA is newer, and -1 if entryB is newer.

const comparison = drive.compare(entryA, entryB)

drive.watch([folder])

src

Returns an async iterator that watches folder (defaults to /) and yields [current, previous] snapshot pairs whenever the drive changes. The snapshots are auto-closed before the next value, so do not close them yourself.

• Returns: Watcher — an iterator that listens on folder to yield changes, by default on /.

for await (const [current, previous] of watcher) {
  console.log(current.version)
  console.log(previous.version)
}

Versions and batched mutations

drive.batch()

src

Useful for atomically mutate the drive, has the same interface as Hyperdrive.

• Returns: Hyperdrive — A batch with the same interface as Hyperdrive; call batch.flush() to commit.

const batch = drive.batch()
await batch.put('/a.txt', Buffer.from('a'))
await batch.flush()

await batch.flush()

src

Commit a batch of mutations to the underlying drive.

• Returns: Promise<void> — Resolves once the batch is committed and closed.

await batch.flush()

drive.checkout(version)

src

Get a read-only snapshot of a previous version.

• Returns: Hyperdrive — A read-only Hyperdrive at the requested version.

const snapshot = drive.checkout(4)

drive.diff(version, folder, [options])

src

Efficiently create a stream of the changes to folder between version and drive.version.

• Returns: Readable — A stream of { left, right } diff entries.

await drive.truncate(version, [options] })

src

Truncates the Hyperdrive to a previous version (both the file-structure reference and the blobs).

• Returns: Promise<void> — Resolves once the drive and its blobs are truncated.
• Throws: BAD_ARGUMENT if the truncation length is invalid.

await drive.truncate(drive.version - 1)

Replication, downloads, and blob access

drive.mirror(out, [options])

src

Efficiently mirror this drive into another. Returns a MirrorDrive instance constructed with options.

• Returns: MirrorDrive — A MirrorDrive instance; await it or mirror.done() to finish.

const mirror = drive.mirror(out)
await mirror.done()

drive.download(folder, [options])

src

Downloads the blobs corresponding to all entries in the drive at paths prefixed with folder. Returns a Download object that resolves once all data has been downloaded:

• Returns: Download — A Download object; await download.done() to know when complete.

const download = await drive.download(key)
await download.done()

download.destroy()

await drive.downloadDiff(version, folder, [options])

src

Downloads all the blobs in folder corresponding to entries in drive.checkout(version) that are not in drive.version. Returns a Download object that resolves once all data has been downloaded:

• Returns: Download — A Download object; await download.done() to know when complete.

const download = await drive.downloadDiff(version, folder)
await download.done()

download.destroy()

await drive.downloadRange(dbRanges, blobRanges)

src

Downloads the entries and blobs stored in the ranges dbRanges and blobRanges. Returns a Download object that resolves once all data has been downloaded:

• Returns: Download — A Download object; await download.done() to know when complete.

const download = await drive.downloadRange(dbRanges, blobRanges)
await download.done()

download.destroy()

drive.findingPeers()

src

Indicate to Hyperdrive that you're finding peers in the background, requests will be on hold until this is done.

• Returns: Function — A done callback to call once peer discovery has finished.

const done = drive.findingPeers()
swarm.flush().then(done, done)

drive.replicate(isInitiatorOrStream)

src

Creates a replication stream for the drive. Pass true/false to create a new stream as initiator/responder, or pass an existing stream or socket to replicate over it. See corestore.replicate for how replication works.

• Returns: Stream — A replication stream.

const swarm = new Hyperswarm()
const done = drive.findingPeers()
swarm.on('connection', (socket) => drive.replicate(socket))
swarm.join(drive.discoveryKey)
swarm.flush().then(done, done)

await drive.update([options])

src

Waits for initial proof of the new drive version until all findingPeers are done.

• Returns: Promise<boolean> — true if the drive advanced to a new version, otherwise false.

await drive.getBlobs()

src

Returns the Hyperblobs instance storing the blobs indexed by drive entries.

• Returns: Promise<Hyperblobs> — the Hyperblobs instance storing the blobs indexed by drive entries.

await drive.put('/file.txt', Buffer.from('hi'))

const buffer1 = await drive.get('/file.txt')

const blobs = await drive.getBlobs()
const entry = await drive.entry('/file.txt')
const buffer2 = await blobs.get(entry.value.blob)

// => buffer1 and buffer2 are equals

await drive.getBlobsLength(checkout)

src

Returns the length of the Hyperblobs instance at the time of the specified Hyperdrive version (defaults to the current version).

• Returns: Promise<number> — the length of the Hyperblobs instance at the time of the specified Hyperdrive version (defaults to the current version).

const blobsLength = await drive.getBlobsLength()

Types

DriveOptions

Options for creating a Hyperdrive.

TruncateOptions

Options for drive.truncate().

UpdateOptions

Options for drive.update().

GetOptions

Options for drive.get().

WriteOptions

Options for writing a file (drive.put() and drive.createWriteStream()).

ClearOptions

Options for clearing blobs (drive.clear() and drive.clearAll()).

SymlinkOptions

Options for drive.symlink().

EntryOptions

Options for drive.entry().

Errors

Coded errors this module can throw — catch them via err.code.

See also

• Create a full peer-to-peer filesystem with Hyperdrive—end-to-end filesystem replication walkthrough.
• Work with many Hypercores using Corestore—recommended pattern when one app manages multiple drives or related cores.
• Localdrive—map a Hyperdrive-like API onto the local filesystem.
• Mirrordrive—sync between Hyperdrive and other drive-like destinations.
• Corestore—shared storage and replication manager for drive metadata and content.
• Hyperbee—Hyperdrive uses a Hyperbee internally for metadata indexing.
• Drives—CLI tool for creating, mirroring, and seeding Hyperdrives.