Automerge
Automerge is a library of data structures for building collaborative applications in JavaScript.
A common approach to building JavaScript apps involves keeping the state of your application in
model objects. For example, imagine you are developing a task-tracking app in which each task is
represented by a card. In vanilla JavaScript you might write the following:
var state = {cards: []}
state.cards.push({title: 'Reticulate splines', done: false})
state.cards[0].done = true
localStorage.setItem('MyToDoList', JSON.stringify(state))
Automerge is used in a similar way, but the big difference is that it supports automatic syncing
and merging:
-
You can have a copy of the application state locally on several devices (which may belong to the
same user, or to different users). Each user can independently update the application state on
their local device, even while offline, and save the state to local disk.
(Similar to git, which allows you to edit files and commit changes offline.)
-
When a network connection is available, Automerge figures out which changes need to be synced from
one device to another, and brings them into the same state.
(Similar to git, which lets you push your own changes, and pull changes from other developers,
when you are online.)
-
If the state was concurrently changed on different devices, Automerge automatically merges the
changes together cleanly, so that everybody ends up in the same state, and no changes are lost.
(Different from git: no merge conflicts to resolve!)
Features and Design Principles
- Network-agnostic. Automerge is a pure data structure library that does not care what kind of
network you use: client/server, peer-to-peer, Bluetooth, carrier pigeon, whatever, anything goes.
Bindings to particular networking technologies are handled by separate libraries. For example, see
MPL for an implementation that uses Automerge in a
peer-to-peer model using WebRTC.
- Immutable state. A Automerge object is an immutable snapshot of the application state at one
point in time. Whenever you make a change, or merge in a change that came from the network, you
get back a new state object reflecting that change. This fact makes Automerge compatible with the
functional reactive programming style of Redux and
Elm, for example. Internally, Automerge is built upon Facebook's
Immutable.js, but the Automerge API uses regular
JavaScript objects (using
Object.freeze
to prevent accidental mutation). - Automatic merging. Automerge is a so-called Conflict-Free Replicated Data Type
(CRDT), which allows
concurrent changes on different devices to be merged automatically without requiring any central
server. It is based on academic research on JSON CRDTs, but
the details of the algorithm in Automerge are different from the JSON CRDT paper, and we are
planning to publish more detail about it in the future.
- Fairly portable. We're not yet making an effort to support old platforms, but we have tested
Automerge in Node.js, Chrome, Firefox, and Electron.
Example Usage
const Automerge = require('automerge')
let state1 = Automerge.init()
state1 = Automerge.changeset(state1, 'Initialize card list', doc => {
doc.cards = []
})
state1 = Automerge.changeset(state1, 'Add card', doc => {
doc.cards.push({title: 'Rewrite everything in Clojure', done: false})
})
state1 = Automerge.changeset(state1, 'Add another card', doc => {
doc.cards[1] = {title: 'Reticulate splines', done: false}
})
state1 = Automerge.changeset(state1, 'Add a third card', doc => {
doc.cards.insertAt(0, {title: 'Rewrite everything in Haskell', done: false})
})
let state2 = Automerge.init()
state2 = Automerge.merge(state2, state1)
state1 = Automerge.changeset(state1, 'Mark card as done', doc => {
doc.cards[0].done = true
})
state2 = Automerge.changeset(state2, 'Delete card', doc => {
delete doc.cards[1]
})
state1 = Automerge.merge(state1, state2)
Automerge.getHistory(state1)
.map(state => [state.changeset.message, state.snapshot.cards.length])
For an example of a real-life application built upon Automerge, check out
Trellis, a project management tool.
Setup
If you're in Node.js, you can install Automerge through npm, and then import it with
require('automerge')
as in the example above:
$ npm install --save automerge
Otherwise, clone this repository, and then you can use the following commands:
npm install
— install dependencies.npm test
— run the test suite in Node.npm run browsertest
— run the test suite in web browsers.npm run webpack
— create a bundled JS file for web browsers (including dependencies) that
you can load through a script tag, and write it to dist/automerge.js
.
Caveats
The project currently has a number of limitations that you should be aware of:
- No integrity checking: if a buggy (or malicious) device makes corrupted edits, it can cause
the application state on other devices to be come corrupted or go out of sync.
- No security: there is currently no encryption, authentication, or access control.
- Performance: it's good enough for small applications with a few dozen objects, but there is more
work to be done before it's suitable for more ambitious apps.
- Small number of collaborators: Automerge is designed for small-group collaborations. While there
is no hard limit on the number of devices that can update a document, performance will degrade
if you go beyond, say, 100 devices or so.
- ...and more, see the open issues.
Meta
Copyright 2017, Ink & Switch LLC, and University of Cambridge.
Released under the terms of the MIT license (see LICENSE
).
Created by
Martin Kleppmann,
Orion Henry,
Peter van Hardenberg,
Roshan Choxi, and
Adam Wiggins.