Autobase

Autobase is a higher-level composition over underlying Hypercores. Writers append causal nodes to local cores, Autobase linearizes that graph into an eventually consistent order, and your apply handler materializes a deterministic view, often into a Hyperbee.

The apply handler

Treat apply like a pure reducer: given an ordered batch of nodes and a view handle, derive the next view state deterministically. Mutate only the view passed into apply. Do not:

• read or write external globals,
• open network connections, or
• assume ordering that Autobase has not yet committed. Side effects belong outside the linearization path.

Autobase can reorder previously seen nodes when new causal information arrives. If apply is non-deterministic, different peers will diverge. Writer management (host.addWriter, indexer flags) and migration between apply versions are advanced topics—keep them inside apply only when they operate on the provided view and host handles.

Install

npm i autobase

Quickstart

This minimal flow opens a view, appends one record, updates the linearizer, and reads the materialized value back from base.view.

import Corestore from 'corestore'
import Autobase from 'autobase'

const store = new Corestore('./autobase-demo')
const base = new Autobase(store, null, {
  open: (viewStore) => viewStore.get({ name: 'messages', valueEncoding: 'json' }),
  apply: async (nodes, view) => {
    for (const { value } of nodes) await view.append(value)
  }
})

await base.ready()
await base.append({ type: 'message', text: 'hello from Autobase' })
await base.update()

console.log(await base.view.get(0))
// { type: 'message', text: 'hello from Autobase' }

Autobase can reorder previously seen nodes as new causal information arrives. Keep open and apply deterministic, derive state from the provided store, and mutate only the provided view inside apply.

API Reference

Constructor and lifecycle

new Autobase(store, bootstrap, opts)

src

Instantiate an Autobase.

const base = new Autobase(store, bootstrap, {
  optimistic: true,
  async apply(nodes, view, host) {
    for (const node of nodes) {
      const { value } = node
      // Verify block
      if (value.password !== 'secrets') continue

      // Acknowledge only even numbers
      if (value.num % 2 === 0) await host.ackWriter(node.from.key)

      await view.append(value.num)
    }
  }
})

// Passing the password 'secrets' represents being verifiable
await base.append({ num: 3, password: 'secrets' }, { optimistic: true }) // will not be applied because `ackWriter` isnt called
await base.append({ num: 2, password: 'secrets' }, { optimistic: true }) // will be applied
await base.append({ num: 4, password: 'incorrect' }, { optimistic: true }) // will not be applied because the block isn't verifiable

When optimistic is enabled, validate optimistic blocks inside apply before calling host.ackWriter(key). Otherwise any peer that can write to the underlying network path could attempt to inject values.

await base.ready()

Resolves once the base and its view are open and ready to use.

State and derived view

base.view

src

The view of the autobase derived from writer inputs. The view is created in the open handler and can have any shape. The most common view is a hyperbee.

• Returns: *

base.key

src

The primary key of the autobase.

• Returns: Buffer

base.discoveryKey

src

The discovery key associated with the autobase.

• Returns: Buffer

base.isIndexer

src

Whether the instance is an indexer.

• Returns: boolean

base.writable

src

Whether the instance is a writer for the autobase.

• Returns: boolean

base.length

src

The length of the system core. This is neither the length of the local writer nor the length of the view. The system core tracks the autobase as a whole.

• Returns: number

base.signedLength

src

The index of the system core that has been signed by a quorum of indexers. The system up until this point will not change.

• Returns: number

base.paused

src

Whether the base is currently paused.

• Returns: boolean — true if the autobase is currently paused, otherwise returns false.

Writes and linearization

await base.append(value, opts)

src

Append a new entry to the autobase.

• Returns: Promise<number> — Resolves to the new core length once the value (or batch) has been appended.
• Throws:
  • if the autobase is closing.
  • if the local writer is not active/writable.

await base.update()

src

Fetch all available data and update the linearizer.

• Returns: Promise<void> — Resolves once the linearizer has advanced.

await base.update()

await base.ack(bg = false)

src

Manually acknowledge the current state by appending a null node that references known head nodes. null nodes are ignored by the apply handler and only serve as a way to acknowledge the current linearized state. Only indexers can ack.

• Returns: Promise<void> — Resolves once the ack has been handled.

await base.ack()

Only indexers can acknowledge. An ack appends a null node that references the known heads so peers can converge faster without changing the view.

base.heads()

src

Gets the current writer heads. A writer head is a node which has no causal dependents, aka it is the latest write. If there is more than one head, there is a causal fork which is pretty common.

• Returns: Array<object> — The head nodes (empty until the base is ready).

const heads = base.heads()

await base.pause()

src

Pauses the autobase, preventing the next apply from running until resume() is called.

• Returns: void

base.pause()

await base.resume()

src

Resumes a paused autobase and will check for an update.

• Returns: void

base.resume()

base.setBigBatches(enable = true)

src

Set the autobase to enable or disable big batches. Big batches allow autobase to run the apply function on more blocks at once at the expense of being slower and less responsive.

• Returns: void

base.setBigBatches(true)

Big batches let Autobase call apply with more nodes at once, trading responsiveness for larger deterministic batches.

Replication and metadata

await base.hash()

src

Merkle-tree hash of the system core at its current length.

• Returns: Promise<Buffer> — the hash of the system core's merkle tree roots.

const hash = await base.hash()

base.replicate(isInitiator || stream, opts)

src

Creates a replication stream for replicating the autobase. Arguments are the same as corestores's .replicate().

• Returns: Stream — The replication stream.

const swarm = new Hyperswarm()

// Join a topic
swarm.join(base.discoveryKey)

swarm.on('connection', (connection) => base.replicate(connection))

await base.setUserData(key, value)

src

Sets the User Data value for the provided key. key is a string. value can be either a string or a buffer.

• Returns: Promise<void> — Resolves once written.

await base.setUserData('nickname', 'alice')

await base.getUserData(key)

src

Read a local-only value previously stored with setUserData().

• Returns: Promise<Buffer\|null> — the User Data value for the provided key.

const nickname = await base.getUserData('nickname')

Static helpers

Autobase.getLocalCore(store, handlers, encryptionKey)

src

Generate a local core to be used for an Autobase.

• Returns: Hypercore — The local writer core.

const core = Autobase.getLocalCore(store)

await Autobase.getUserData(core)

src

Get user data associated with an autobase core. referrer is the .key of the autobase the core is from. view is the name of the view.

• Returns: Promise<{referrer: Buffer, view: string\|null}> — The referrer and view name.

const { referrer, view } = await Autobase.getUserData(core)

await Autobase.isAutobase(core, opts)

src

Detect whether a core's first block is an Autobase oplog message.

• Returns: Promise<boolean> — whether the core is an autobase core.

const isBase = await Autobase.isAutobase(core)

View store helper

store.get(name || { name, valueEncoding })

Load a Hypercore by name (passed as name). name should be passed as a string.

• Throws: if no name is provided.

This store instance is the AutoStore passed into open, and it is the intended place to create the cores that make up your deterministic view.

Apply host calls

await host.addWriter(key, { indexer = true })

Add a writer with the given key to the autobase allowing their local core to append. If indexer is true, it will be added as an indexer.

• Throws: if called on the read-only public view.

await host.removeWriter(key)

Remove a writer from the autobase. This will throw if the writer cannot be removed.

• Throws: if called on the read-only public view, or if it would remove the last indexer.

This throws when the writer cannot be removed, such as when it would leave the system without a removable indexer set.

await host.ackWriter(key)

Acknowledge a writer even if they haven't been added before. This is most useful for applying optimistic blocks from writers that are not currently a writer.

• Throws: if called on the read-only public view.

host.interrupt(reason)

Interrupt the applying of writer blocks optionally giving a reason. This will emit an interrupt event passing the reason to the callback and close the autobase.

• Throws: if called on the read-only public view.

Interrupting closes the Autobase and emits the interrupt event so the application can upgrade its apply logic or recover intentionally.

host.removeable(key)

src

Whether the writer with key can be removed without leaving the system without a removable indexer set.

• Returns: boolean — whether the writer for the given key can be removed.

if (base.removeable(writerKey)) await host.removeWriter(writerKey)

The last indexer cannot be removed.

Events

base.on('update', () => { ... })

src

Triggered when the autobase view updates after apply has finished running.

base.on('interrupt', (reason) => { ... })

src

Triggered when host.interrupt(reason) is called in the apply handler. See host.interrupt(reason) for when interrupts are used.

base.on('fast-forward', (to, from) => { ... })

src

Triggered when the autobase fast forwards to a state already with a quorum. to and from are the .signedLength after and before the fast forward respectively.

base.on('is-indexer', () => { ... })

src

Triggered when the autobase instance is an indexer.

base.on('is-non-indexer', () => { ... })

src

Triggered when the autobase instance is not an indexer.

base.on('writable', () => { ... })

Triggered when the autobase instance is now a writer.

base.on('unwritable', () => { ... })

Triggered when the autobase instance is no longer a writer.

base.on('warning', (warning) => { ... })

Triggered when a warning is triggered.

base.on('error', (err) => { ... })

src

Triggered when an error is triggered while updating the autobase.

Types

AppendOptions

Options for base.append().

See also

• Work with many Hypercores using Corestore—manage the groups of cores that Autobase coordinates.
• Corestore—storage and replication manager typically used for Autobase system, writer, and view cores.
• Hypercore—append-only log primitive that Autobase writers build on.
• Hyperbee—common materialized-view target for deterministic indexed state.