@webqit/observer

The Observer API

Motivation • Overview • Polyfill • Design Discussion • Getting Involved • License

Observe and intercept operations on arbitrary JavaScript objects and arrays using a utility-first, general-purpose reactivity API! This API re-explores the unique design of the Object.observe() API and extends it with the best of JavaScript's other reflection and interception APIs - Proxies, accessors - to support any kind of reactive programming model!

Observer API is an upcoming proposal!

Motivation

Tracking mutations on JavaScript objects has historically relied on "object wrapping" techniques with ES6 Proxies, and on "property mangling" techniques with getters and setters. Besides the object identity problem of the first and the object compromissory nature of the second, there is also the "scalability" issue inherent to the techniques and much "inflexibility" in the programming model they enable:

• Scalability: objects have to be created a certain way, or be purpose-built for the specific technique, to participate in the reactivity system; objects you don't own have to be altered in some way - where that's even possible - to be onboarded into the reactivity system. Scalability is inhibited as we must fulfill the implementation criteria for as many objects as will be needed in the design - clamped to the finite number of objects that can be made to work this way!

• Programming model: proxy traps and object accessors only lend themselves to being wired to one underlying listenining logic in the entire program. Objects are effectively open to multiple interactions on the outside but "locked" to one observer on the inside, enabling just a "many to one" communication model. This does not correctly reflect the most common usecases where the idea is to have any number of listeners per event, to enable a "many to many" model! It takes yet a non-trivial amount of effort to go from the default model to the one desired.

Interestingly, we at one time had an object observability primitive that checked all the boxes and touched the very pain points we have today: the Object.observe() API. So, how about an equivalent API that brings all of the good thinking from Object.observe() together with the idea of Proxies, accessors, and JavaScript's other reflection API in one design, delivered as one utility for all things reactivity? This is the idea with the new Observer API!

└ See more in the introductory blog post^draft

An Overview

The Observer API comes as a set of utility functions.

• Method: Observer.observe()
  • Usage
  • Concept: Mutation APIs
  • Concept: Paths
  • Concept: Batch Mutations
  • Concept: Custom Details
  • Concept: Diffing
• Method: Observer.intercept()
  • Usage
• API Reference

Note
This is documentation for Observer@2.x. (Looking for Observer@1.x?)

Method: Observer.observe()

Observe mutations on any object or array!

// Signature 1
Observer.observe( obj, callback[, options = {} ]);

// Signature 2
Observer.observe( obj, props, callback[, options = {} ]);

Usage

Observe arbitrary objects and arrays:

// An object
const obj = {};
// Mtation observer on an object
const abortController = Observer.observe( obj, handleChanges );

// An array
const arr = [];
// Mtation observer on an array
const abortController = Observer.observe( arr, handleChanges );

Now changes will be delivered synchronously - as they happen. (The sync design is discussed shortly.)

// The change handler
function handleChanges( mutations ) {
    mutations.forEach( mutation => {
        console.log( mutation.type, mutation.key, mutation.value, mutation.oldValue );
    } );
}

--> Stop observing at any time by calling abort() on the returned abortController...

// Remove listener
abortController.abort();

...or you can provide your own Abort Signal instance:

// Providing an AbortSignal
const abortController = new AbortController;
Observer.observe( obj, mutations => {
    // Handle...
}, { signal: abortController.signal } );

// Abort at any time
abortController.abort();

Concept: Mutation APIs

Programmatically mutate properties of an object using the Reflect-like set of operators; each operation will be reported by observers:

// A single "set" operation on an object
Observer.set( obj, 'prop0', 'value0' );
Observer.defineProperty( obj, 'prop1', { get: () => 'value1' } );
Observer.deleteProperty( obj, 'prop2' );

// A single "set" operation on an array
Observer.set( arr, 0, 'item0' ); // Array [ 'item0' ]
Observer.deleteProperty( arr, 0 ); // Array [ <1 empty slot> ]

Polyfill limitations

Beware non-reactive operations:

// Literal object operators
delete obj.prop0;
obj.prop3 = 'value3';

// Array methods
arr.push( 'item3' );
arr.pop();

--> Enable reactivity on specific properties with literal object accessors - using the Observer.accessorize() method:

// Accessorize all current enumerable properties
Observer.accessorize( obj );
// Accessorize specific properties (existing or new)
Observer.accessorize( obj, [ 'prop0', 'prop1', 'prop2' ] );

// Make reactive UPDATES
obj.prop0 = 'value0';
obj.prop1 = 'value1';
obj.prop2 = 'value2';

// Accessorize all current indexes
Observer.accessorize( arr );
// Accessorize specific indexes (existing or new)
Observer.accessorize( arr, [ 0, 1, 2 ] );

// Make reactive UPDATES
arr[ 0 ] = 'item0';
arr[ 1 ] = 'item1';
arr[ 2 ] = 'item2';

// Bonus reactivity with array methods that re-index existing items
arr.unshift( 'new-item0' );
arr.shift();

Polyfill limitations

Beware non-reactive operations:

// The delete operator and object properties that haven't been accessorized
delete obj.prop0;
obj.prop3 = 'value3';

// Array methods that do not re-index existing items
arr.push( 'item0' );
arr.pop();

--> Enable reactivity on arbitray properties with Proxies - using the Observer.proxy() method:

// Obtain a reactive Proxy for an object
const $obj = Observer.proxy( obj );

// Make reactive operations
$obj.prop1 = 'value1';
$obj.prop4 = 'value4';
$obj.prop8 = 'value8';

// With the delete operator
delete $obj.prop0;

// Obtain a reactive Proxy for an array
const $arr = Observer.proxy( arr );

// Make reactive operations
$arr[ 0 ] = 'item0';
$arr[ 1 ] = 'item1';
$arr[ 2 ] = 'item2';

// With an instance method
$arr.push( 'item3' );

And no problem if you end up nesting the approaches.

// 'value1'-->obj
Observer.accessorize( obj, [ 'prop0', 'prop1', 'prop2', ] );
obj.prop1 = 'value1';

// 'value1'-->$obj-->obj
let $obj = Observer.proxy( obj );
$obj.prop1 = 'value1';

// 'value1'-->set()-->$obj-->obj
Observer.set( $obj, 'prop1', 'value1' );

--> "Restore" accessorized properties to their normal state by calling the unaccessorize() method:

Observer.unaccessorize( obj, [ 'prop1', 'prop6', 'prop10' ] );

--> "Reproduce" original objects from Proxies obtained via Observer.proxy() by calling the unproxy() method:

obj = Observer.unproxy( $obj );

Concept: Paths

Observe "a value at a path" on a given tree:

// A tree structure that satisfies the path above
const obj = {
  level1: {
    level2: 'level2-value',
  },
};

const path = Observer.path( 'level1', 'level2' );
Observer.observe( obj, path, m => {
  console.log( m.type, m.path, m.value, m.isUpdate );
} );

Observer.set( obj.level1, 'level2', 'level2-new-value' );

Console

type	path	value	isUpdate
set	[ level1, level2, ]	level2-new-value	true

And well, the initial tree structure can be whatever:

// A tree structure that is yet to be built
const obj = {};

const path = Observer.path( 'level1', 'level2', 'level3', 'level4' );
Observer.observe( obj, path, m => {
  console.log( m.type, m.path, m.value, m.isUpdate );
} );

Now, any operation that "modifies" the observed tree - either by extension or truncation - will fire our listener:

Observer.set( obj, 'level1', { level2: {}, } );

Console

type	path	value	isUpdate
set	[ level1, level2, level3, level4, ]	undefined	false

Meanwhile, this next one completes the tree, and the listener reports a value at its observed path:

Observer.set( obj.level1, 'level2', { level3: { level4: 'level4-value', }, } );

Console

type	path	value	isUpdate
set	[ level1, level2, level3, level4, ]	level4-value	false

If you were to find the exact point of mutation in the path in an audit trail, you could inspect the event's context property, which itself returns the parent event...

let context = m.context;

...allowing you to go up one more level until the root event:

let parentContext = context.context;

Where a promise is encountered along the path, further access is paused until promise resolves:

Observer.set( obj.level1, 'level2', Promise.resolve( { level3: { level4: 'level4-value', }, } ) );

Concept: Batch Mutations

Make multiple mutations at a go, and they'll be correctly delivered in batch to observers!

// Batch operations on an object
Observer.set( obj, {
    prop0: 'value0',
    prop1: 'value1',
    prop2: 'value2',
} );
Observer.defineProperties( obj, {
    prop0: { value: 'value0' },
    prop1: { value: 'value1' },
    prop2: { get: () => 'value2' },
} );
Observer.deleteProperties( obj, [ 'prop0', 'prop1', 'prop2' ] );

// Batch operations on an array
Observer.set( arr, {
    '0': 'item0',
    '1': 'item1',
    '2': 'item2',
} );
Object.proxy( arr ).push( 'item3', 'item4', 'item5', );
Object.proxy( arr ).unshift( 'new-item0' );
Object.proxy( arr ).splice( 0 );

--> Use the Observer.batch() to batch multiple arbitrary mutations - whether related or not:

Observer.batch( arr, async () => {
    Observer.set( arr, 0, 'item0' ); // Array [ 'item0' ]
    await somePromise();
    Observer.set( arr, 2, 'item2' ); // Array [ 'item0', <1 empty slot>, 'item2' ]
} );

Method calls on a proxied instance - e.g. Object.proxy( arr ).splice( 0 ) - also follow this strategy.

Concept: Custom Details

Pass some custom detail - an arbitrary value - to observers via a params.detail property.

// A set operation with detail
Observer.set( obj, {
    prop2: 'value2',
    prop3: 'value3',
}, { detail: 'Certain detail' } );

Observers recieve this value on their mutation.detail property.

// An observer with detail
Observer.observe( obj, 'prop1', mutation => {
    console.log( 'A mutation has been made with detail:' + mutation.detail );
} );

Concept: Diffing

Receive notifications only for mutations that actually change property state, and ignore those that don't.

// Responding to state changes only
Observer.observe( obj, handleChanges, { diff: true } );

// Recieved
Observer.set( obj, 'prop0', 'value' );

// Ignored
Observer.set( obj, 'prop0', 'value' );

Method: Observer.intercept()

Intercept operations on any object or array before they happen!

// Signature 1
Observer.intercept( obj, prop, handler[, options = {} ]);

// Signature 2
Observer.intercept( obj, traps[, options = {} ]);

Usage

Extend standard operations on an object - Observer.set(), Observer.deleteProperty(), etc - with custom traps using the Observer.intercept() method!

Below, we intercept all "set" operations for an HTTP URL then transform it to an HTTPS URL.

const setTrap = ( operation, previous, next ) => {
    if ( operation.key === 'url' && operation.value.startsWith( 'http:' ) ) {
        operation.value = operation.value.replace( 'http:', 'https:' );
    }
    return next();
};
Observer.intercept( obj, 'set', setTrap );

Now, only the first of the following will fly as-is.

// Not transformed
Observer.set( obj, 'url', 'https://webqit.io' );

// Transformed
Observer.set( obj, 'url', 'http://webqit.io' );

And below, we intercept all "get" operations for a certain value to trigger a network fetch behind the scenes.

const getTrap = ( operation, previous, next ) => {
    if ( operation.key === 'token' ) {
        return next( fetch( tokenUrl ) );
    }
    return next();
};
Observer.intercept( obj, 'get', getTrap );

And all of that can go into one "traps" object:

Observer.intercept( obj, {
    get: getTrap,
    set: setTrap,
    deleteProperty: deletePropertyTrap,
    defineProperty: definePropertyTrap,
    ownKeys: ownKeysTrap,
    has: hasTrap,
    // etc
} );

The Polyfill

Use as an npm package:

npm i @webqit/observer

// Import
import Observer from '@webqit/observer';;

Use as a script:

<script src="https://unpkg.com/@webqit/observer/dist/main.js"></script>

4.4 kB min + gz | 13.9 KB min ↗

// Obtain the APIs
const Observer = window.webqit.Observer;

API Reference

A unifying API over related but disparate things like Object.observe(), Reflect APIs, and the "traps" API!

Observer API	Reflect API	Trap
apply() ↗	✓	apply() {}
batch() ↗	×	-
construct() ↗	✓	construct() {}
defineProperty() ↗	✓	defineProperty() {}
deleteProperty() ↗	✓	deleteProperty() {}
get() ↗	✓	get() {}
getOwnPropertyDescriptor() ↗	✓	getOwnPropertyDescriptor() {}
getPrototypeOf() ↗	✓	getPrototypeOf() {}
has() ↗	✓	has() {}
intercept()↗	×	-
isExtensible() ↗	✓	isExtensible() {}
observe() ↗	×	-
ownKeys() ↗	✓	ownKeys() {}
path() ↗	×	-
preventExtensions() ↗	✓	preventExtensions() {}
set() ↗	✓	set() {}
setPrototypeOf() ↗	✓	setPrototypeOf() {}
.	.	.
accessorize() ↗	×	-
proxy() ↗	×	-

Design Discussion

[TODO]

Getting Involved

All forms of contributions are welcome at this time. For example, implementation details are all up for discussion. And here are specific links:

• Project
• Documentation
• Discusions
• Issues

License

MIT.