derivable

DerivableJS

Observable State done Right

DerivableJS is a JavaScript implementation of Derivables.

• Derivables
  • Tradeoffs
• Usage - API - Debugging - Examples (very wip) - npm - Browser - Batteries Not Included - Equality Woes
• 1.0.0 Roadmap
• Future Work
• Contributing
• Thanks
• License

Derivables

Derivables make it trivial to maintain consistent (i.e. sense-making) state at all times without requiring that it be kept all in one place. This is a huge win for those of us who develop complex systems with lots of moving parts because it eradicates an entire class of subtle-but-devastating bugs along with all the incidental complexity they fed upon, allowing us to spend more quality time getting intimate with our problem domain.

This library satisfies the notion that changes in state should not cause state changes, i.e. if the value of state A depends on the value of state B, updates to B should atomically include updates to A—they should be the same update. We don't seem to have a handle on this issue, and it causes serious mess in our brains and code.

Derivables clean that mess up by enabling you to make elegant declarative statements about how your bits of state are related. Then, when you update any bits of 'root' state, clever computer-sciency stuff happens in order to keep everything—every goshdarn thing—consistent 100% of the time.

There are two types of Derivable:

• Atoms are simple references to immutable values. They are the roots; the ground truth from which all else is derived.
• Derivations represent pure (as in function) transformation of values held in atoms.

Changes in atoms or derivations can be monitored by Reactors, which do not encapsulate values and exist solely for executing side-effects in reaction to state changes. Reactors can also be stopped and restarted when appropriate, and offer lifecycle hooks for the sake of resource management.

Let's have a look at a tiny example app which greets the user:

import {atom, derive, transact} from 'derivable'

// global application state
const name = atom("World");     // the name of the user  
const countryCode = atom("en"); // for i18n

// static constants don't need to be wrapped
const greetings = {
  en: "Hello",
  de: "Hallo",
  es: "Hola",
  cn: "您好",
  fr: "Bonjour",
};

// derive a greeting message based on the user's name and country.
const greeting = countryCode.derive(cc => greetings[cc]);
const message = derive`${greeting}, ${name}!`; // es6 tagged template strings!

// set up a Reactor to print the message every time it changes
message.react(msg => console.log(msg));
// $> Hello, World!

countryCode.set("de");
// $> Hallo, World!
name.set("Dagmar");
// $> Hallo, Dagmar!

// we can avoid unwanted intermediate reactions by using transactions
transact(() => {
  countryCode.set("fr");
  name.set("Étienne");
});
// $> Bonjour, Étienne!

The structure of this example can be depicted as the following DAG:

The DAG edges are automatically inferred by DerivableJS. It is important to understand that they (the edges) do not represent data flow in any temporal sense. They are not streams or channels or even some kind of callback chain. When you change the value of an atom, its whole propagation graph updates in atomic accord. There is no accessible point in time between the fact of changing an atom and the fact of its dependents becoming aware of the change.

To put it another way: the (atoms + derivations) part of the graph is conceptually a single gestalt reference to a value. In this case the value is a virtual composite of the two atoms' states. The individual nodes are merely views into this value; they constitute the same information presented differently, like light through a prism. The gestalt is always internally consistent no matter which specific parts of it you inspect at any given time.

This property is super important and useful. It cannot be replicated with Observables or any other callback-based mechanism (without doing extra stuff involving topological sorting, and even then only in a single threaded environment).

The other thing which truly sets derivations apart is that they are totally lazy. Like values in Haskell they are computed just-in-time, i.e. on demand. This is another huge win because:

• It decouples the computational complexity of updating atoms with that of computing their derivations. Derivations are only re-computed at atom-change time if they (the derivations) are actually used by an affected reactor. So, for example, you can declare an eternal relationship between n and n² without needing to fear the cost of re-computing n² every time n changes. That fear is transferred to whoever decides that they want to know the value of n² at all times, which is just how it should be.
• It allows derivations to be automatically garbage collected when you don't need them any more, just like any other object. This is simple to the max! In fact, you don't need any special knowledge to avoid memory leaks with DerivableJS—it Just Works.
• It permits true short-circuiting boolean logic in derivation structures, which turns out to be extraordinarily practical.

Tradeoffs

You may be wondering how these benefits are achieved. The answer is simple: mark-and-sweep. Yes, just like your trusty Garbage Collectors have been doing since the dawn of Lisp. It is actually more like mark-react-sweep, and it brings a couple of performance hits over streams, channels, and callback chains:

• When an atom is changed, its entire derivation graph is traversed and 'marked'. All active reactors found in the graph are then gently prodded and told to decide whether they need to re-run themselves. This amounts to an additional whole-graph traversal in the worst case. The worst case also happens to be the common case :(
• The sweep phase involves yet another probably-whole-graph traversal.

So really each time an atom is changed, its entire derivation graph is likely to be traversed 3 times. I would argue that this is negligible for most UI-ish use cases. The traversal is really simple stuff: following pointers and doing numeric assignments/comparisons. Computers are stupidly good at that kind of thing. But if you're doing something intense then perhaps DerivableJS isn't the best choice and you should pick something with eager evaluation. Be appraised, however, that I've got a fairly promising idea for how to reduce the traversal overhead after v1.0.0 drops.

Side note: during transactions only the mark phase occurs. And if an atom is changed more than once during a single transaction, only the bits of the derivation graph that get dereferenced between changes are re-marked.

A final potential drawback is that DerivableJS requires one to think and design in terms of pure functions and immutable data being lazily computed, which I think takes a little while to get comfortable with coming directly from an OO background.

Usage

DerivableJS is still quite new, but has been used for serious stuff in production. I think it is safe to consider it beta quality at this point.

API

See Here

Debugging

Due to inversion of control, the stack traces you get when your derivations throw errors can be totally unhelpful. There is a nice way to solve this problem for dev time. See setDebugMode for more info.

Examples (very wip)

The best example of writing good code with Derivables right now is the talk demo, which is presented as a 'diff tutorial' and should be read from the initial commit.

The next best is the routing walkthrough

I've also implemented a solution to @staltz's flux challenge.

And there are a few others here too.

More coming!

npm

Available as derivable.

Browser

Either with browserify or, if need be, import dist/derivable.min.js directly (find it at window.Derivable).

Batteries Not Included

DerivableJS expects you to use immutable (or effectively immutable) data. It also expects derivation functions to be pure. JavaScript isn't really set up to handle such requirements out of the box, so you would do well to look at an FP library like Ramda to make life easier. Also, if you want to do immutable collections properly, Immutable or Mori are probably the way to go. Godspeed!

Equality Woes

JavaScript is entirely whack when it comes to equality. People do crazy jazz trying to figure out if some stuff is the same as some other stuff.

If the data you're threading through DerivableJS needs its own notion of equality, make sure it has a .equals method and everything will be fine.

If you're using a data library with some custom non-standard mechanism for doing equality checks (e.g. Mori), then you'll need to re-initialize DerivableJS with a custom equality function.

import { withEquality } from 'derivable'

const { atom, derive, ..._} = withEquality(myCustomEqualityChecker);

1.0.0 Roadmap

DerivableJS's API will be unstable until version 1.0.0 is released, whereafter the project will use Semantic Versioning.

I plan to wait for the project to pick up a bit more steam so I can get serious community feedback before pumping out a 1.0.0 release. This is to allow for breaking changes if the need arises.

Future Work

1. Shrink the code base. It is currently 5.4k minified and gzipped, but I didn't write the code with size in mind so I think it can get much smaller. now about 3.6k, but could probably get smaller still
2. Dynamic graph optimization. e.g. collapsing derivation branches of frequently-executed reactions into one derivation, maybe trying to align all the data in memory somehow. This would be similar to JIT tracing sans optimization, and could make enormous derivation graphs more feasible (i.e. change propagation could become linear in the number of reactors rather than linear in the number of derivation nodes. It wouldn't work with parent inference though; you'd have to write derivations in the x.derive((x, y, z) => ..., y, z) or derive(x, (x, y, z) => ..., y z) fashions. So do that if you want to get ahead of the curve!
3. Investigate whether asynchronous transactions are possible, or indeed desirable.
4. Investigate debugging support. - implemented in 0.10.0
5. I've got a feeling one of the whole-graph traversals mentioned in Tradeoffs can be eliminated while maintaining all the goodness DerivableJS currently provides, but it would involve a lot of extra caching and it won't even be needed if (1) turns out to be fruitful, so I'll try that first.

Contributing

I heartily welcome questions, feature requests, bug reports, and general suggestions/criticism on the github issue tracker. I also welcome bugfixes via pull request (please read CONTRIBUTING.md before sumbitting).

Thanks

Special thanks to:

• Alan Dipert and Micha Niskin, creators of Javelin (and Boot!). Their talk on Javelin was the first exposure I had to these ideas.
• Michael Thompson for the re-frame README which was an awesome resource and gave me enough enthusiasm for the idea to hunker down and do it.
• David Weir and Jeremy Reffin for their invaluable mentorship.
• Rich Hickey and the Clojure community for being a constant source of ideas and for making programming even more fun.

License

Copyright 2015 David Sheldrick <[e-mail]>

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Changelog

0.12.0

Highlights

• Derivable#derive does more stuff (destructuring, property/index lookup, regex matching)
• Derivable#react now provides options for declarative lifecycle control.
• Composite Lenses
• Fine-grained Equality checks
• Removed a bunch of API cruft.

New Stuff

Declarative Reactor Lifecycle Control

Prior to this release, the lifecycles of Reactors were controlled imperatively using theReactor#start and Reactor#stop methods. This was somewhat verbose and, more importantly, went against the grain of this whole declarative/reactive thing we have going here.

Now there is a better way: providing a lifetime configuration object as a second argument to the .react method. Here are the options:

interface Lifecycle {
  from?: (() => boolean) | Derivable<boolean>;
  when?: (() => boolean) | Derivable<boolean>;
  until?: (() => boolean) | Derivable<boolean>;
  skipFirst?: boolean;
  once?: boolean;
  onStart?: () => void;
  onStop?: () => void;
}

from determines this initialization time of the reactor. i.e. when the given derivable becomes truthy, the reactor is initialized.

when causes .start and .stop to be called when the given derivable becomes truthy and falsey respectively (but not before the reactor has been initialized, and not after it has been killed).

until causes the reactor to be killed when the given derivable becomes truthy.

skipFirst causes the first invocation of the reactor (after it has been initialized, and when when is truthy) to be ignored. This is typically used if the state of the world at the time of declaration is such that invoking the reactor would be redundant or harmful.

once causes the reactor to be killed immediately following its first invocation.

onStart and onStop are the same lifecycle hooks that were previously provided.

Example usage:

const n = atom(0);

n.react(n => console.log(`n is ${n}`), {
  from: () => n.get() > 0,      // start when n > 0
  when: () => n.get() %2 === 0, // only react when n is even
  until: () => n.get() => 5     // stop when n >= 5
});
// ... no output

n.set(1);
// ... no output (n is odd)

n.set(2);
// $> n is 2

n.set(3);
// ... no output (n is odd)

n.set(4);
// $> n is 4

n.set(5);
// ... no output (reactor was killed)

n.set(4);
// ... no output (reactors don't come back from the dead)

Derivable#derive new capabilities

• RegExp matching

  const string = atom('hello world');
  const firstLetters = string.derive(/\b\w/g);
  
  firstLetters.get();
  // => ['h', 'w']
  

• Property/Index lookup

  const obj = atom({foo: 'FOO!'});
  const foo = obj.derive('foo');
  
  foo.get();
  // => 'FOO!'
  
  const arr = atom(['one', 'two']);
  const first = arr.derive(0);
  
  first.get();
  // => 'one'
  

• Destructuring

  const string = atom('hello world')
  const [len, upper, firstChar, words] = string.derive([
    'length', s => s.toUpperCase(), 0, /\w+/g
  ]);
  

Also note that these work with derivable versions of the arguments:

const arr = atom(['one', 'two', 'three']);
const idx = atom(0);
const item = arr.derive(idx);

item.get();
// => 'one'

idx.set(1);
item.get();
// => 'two'

Composite Lenses

Previously 'lensed atoms' could only have one underlying atom. It is now possible to lens over an arbitrary number of atoms using the new CompositeLens interface:

type CompositeLens<T> = {
  // no-arg getter uses lexical closure to deref and combine atoms
  get: () => T,

  // one-arg setter to tease apart the value being set and push it
  // up to the atoms manually
  // runs in an implicit transaction.
  set: (value: T) => void
}

Instances of which may be passed to a new 1-arity version of the top-level lens function to create lensed atoms:

const $FirstName = atom('John');
const $LastName = atom('Steinbeck');

const $Name = lens({
  get: () => $FirstName.get() + ' ' + $LastName.get(),
  set: (val) => {
    const [first, last] = val.split(' ');
    $FirstName.set(first);
    $LastName.set(last);
  }
});

$Name.get(); // => 'John Steinbeck'

$Name.set('James Joyce').

$LastName.get(); // => 'Joyce'

Fine-grained Equality Control

Because JS has no standard way to override equality comparisons, DerivableJS makes it possible to inject equality-checking logic at the module level using the top-level function withEquality which returns a new instance of DerivableJS using the given equality-checking function.

It is now also possible to do this on a per-derivable basis.

The new Derivable#withEquality method creates a clone of a derivable, which new derivable uses the given equality-checking function. It looks like this:

import { equals } from 'ramda'

const $Person = atom({name: "Steve"}).withEquality(equals);
$Person.react(({name}) => console.log(`name is ${name}`));
// $> name is Steve

$Person.set({name: "Steve"});
// ... no output (this would print the name again
// if using DerivableJS's standard equality function
// which only does strict-equality (===) checks if no .equals
// method is present on the arguments being compared)

atomic/atomically

These new top-level functions are identical to transaction/transact respectively except that they do not create new (nested) transactions if already in a transaction. This is almost always the desired behaviour, unless you want to gracefully abort transactions.

Breaking changes:

The Derivable#react method:

• no longer returns a Reactor.
• only accepts functions as the first argument.
• does not bind the given function to the context of the resultant reactor.

You can get the old behaviour by converting

d.react(f);

to

d.reactor(f).start().force();

Although it is recommended to switch to using the new declarative lifecycle stuffs if possible.

The Derivable#reactWhen method was removed

Use $d.react(r, {when: $when}).

Dependent reactors are no longer stopped automatically when their governors are.

That was a silly idea...

The following top-level functions were removed due to cruftiness:

• derive (except the tagged template string version, that's still there). Use the Derivable#derive method instead.
• mDerive. Use the Derivable#mDerive method instead.
• 2+ arity version of lens. Use the Derivable#lens method instead.
• lookup. Use the Derivable#derive(string|number) method instead.
• destruct. Use the Derivable#derive([string|number]) method instead.
• ifThenElse. Use the Derivable#then method instead.
• mIfThenElse. Use the Derivable#mThen method instead.
• not. Use the Derivable#not method instead.
• switchCase. Use the Derivable#switch method instead.
• get. Use the Derivable#get method instead.
• set. Use the Atom#set method instead.
• swap. Use the Atom#swap method instead.