@webqit/stateful-js

Stateful JS

Overview • Creating Stateful Programs • Polyfill • Examples • License

Stateful JS is a runtime extension to JavaScript that enables us do Imperative Reactive Programming (IRP) in the very language! This project pursues a futuristic, more efficient way to build reactive applocations today!

Overview

Whereas you normally would need a couple primitives to model reactive logic...

import { createSignal, createMemo, createEffect } from 'solid-js';

// count
const [ count, setCount ] = createSignal(5);
// doubleCount
const doubleCount = createMemo(() => count() * 2);
// console.log()
createEffect(() => {
  console.log(doubleCount());
});

// An update
setTimeout(() => setCount(10), 500);

Stateful JS lets you acheive the same in the ordinary imperative form of the language:

let count = 5;
let doubleCount = count * 2;
console.log(doubleCount);

// An update
setTimeout(() => count = 10, 500);

Here, the code you write is able to statically reflect changes to state in micro details, such that the state of that piece of program is always in sync with the rest of the program at any given point!

Idea

Show

Imperative programs are really the foundation for "state", "effect" and much of what we try to model today at an abstract level using, sometimes, functional reactive primitives as above, and sometimes some other means to the same end. Now, that's really us re-implementing existing machine-level concepts that should be best left to the machine!

Learn more

Right in how the instructions in an imperative program "act" on data - from the assignment expression that sets or changes the data held in a local variable (count = 10) to the delete operator that mutates some object property (delete object.value), to the "if" construct that determines the program's execution path based on a certain state - we can see all of "state" (data), "effect" (instructions that modify state/data), and control structures (instructions informed by state, in turn) at play!

But what we don't get with how this works naturally is having the said instructions stay sensitive to changes to the data they individually act on! (The runtime simply not maintaining that relationship!) And that's where the problem lies; where a whole new way of doing things becomes necessary - wherein we have to approach literal operations programmatically: setCount(10) vs count = 10.

If we could get the JS runtime to add "reactivity" to how it already works - i.e. having the very instructions stay sensitive to changes to the data they individually act on - we absolutely would be enabling reactive programming in the imperative form of the language and entirely unnecessitating the manual way!

This is what we're exploring with Stateful JS!

Creating Stateful Programs

This feature comes both as a new function type: "Stateful Functions" and as a new execution mode for whole programs: "Stateful Execution Mode" (or "Stateful Mode" for short; just in how we have "Strict Mode")!

Given a language-level feature, no setup or build step is required! Polyfill just ahead!

Stateful Functions

You can designate a function as stateful using a double star notation; similar to how generator functions look:

// Stateful function declaration
function** bar() {
  let count = 5;
  let doubleCount = count * 2;
  console.log(doubleCount);
}
bar();

// Stateful async function declaration
async function** bar() {
  let count = await 5;
  let doubleCount = count * 2;
  console.log(doubleCount);
}
await bar();

...and in just how a function works in JavaScript

// Stateful function expression, optionally async
const bar = function** () {
  // Function body
}

// Stateful object property, optionally async
const foo = {
  bar: function** () {
    // Function body
  },
}

// Stateful object method, optionally async
const foo = {
  **bar() {
    // Function body
  },
}

// Stateful class method, optionally async
class Foo {
  **bar() {
    // Function body
  }
}

And you can acheive the same using Stateful Function constructors:

// Stateful function constructor
const bar = StatefulFunction(`
  let count = 5;
  let doubleCount = count * 2;
  console.log(doubleCount);
`);
bar();

// Stateful async function constructor
const bar = StatefulAsyncFunction(`
  let count = await 5;
  let doubleCount = count * 2;
  console.log(doubleCount);
`);
await bar();

...and in just how function constructors work in JavaScript

// With function parameters
const bar = StatefulFunction( param1, ... paramN, functionBody );

// With the new keyword
const bar = new StatefulFunction( param1, ... paramN, functionBody );

// As class property
class Foo {
  bar = StatefulFunction( param1, ... paramN, functionBody );
}

Well, this also includes the fact that, unlike normal function declarations and expressions that can see their surrounding scope, code in function constructors can see only the global scope:

let a;
globalThis.b = 2;
var c = 'c'; // Equivalent to globalThis.c = 'c' assuming that we aren't running in a function scope or module scope
const bar = StatefulFunction(`
  console.log(typeof a); // undefined
  console.log(typeof b); // number
  console.log(typeof c); // string
`);
bar();

Stateful Execution Mode (Whole Programs)

Think "Strict Mode", but for reactivity!

Here, given the same underlying infrastructure, any piece of code should be able to run in stateful mode. Stateful JS exposes two APIs that enable just that:

// Stateful regular JS
const program = new StatefulScript(`
  let count = 5;
  let doubleCount = count * 2;
  console.log(doubleCount);
`);
program.execute();

// Stateful module
const program = new StatefulModule(`
  let count = await 5;
  let doubleCount = count * 2;
  console.log(doubleCount);
`);
await program.execute();

These will run in the global scope!

The latter does certainly let you use import and export statements!

Exanple

// Stateful module
const program = new StatefulModule(`
  import module1, { module2 } from 'package-name';
  import { module3 as alias } from 'package-name';
  ...
  export * from 'package-name';
  export let localVar = 0;
`);

Now, this goes a step further to let us have "Stateful Scripts" - which ships in a related work OOHTML:

<!-- Stateful classic script -->
<script stateful>
  let count = 5;
  let doubleCount = count * 2;
  console.log(doubleCount);
</script>

<!-- Stateful module script -->
<script type="module" stateful>
  let count = await 5;
  let doubleCount = count * 2;
  console.log(doubleCount);
</script>

And the ideas there are coming to simplify how we build single page applications!

Sneak peak

<main id="page1">
  <script scoped stateful>

    console.log(this.id); // page1

  </script>
</main>

<main id="page2">
  <script type="module" scoped stateful>

    console.log(this.id); // page2

  </script>
</main>

Now, other tooling may choose to use the same infrastructure in other ways; e.g. as compile target.

Consuming Stateful Programs

Each call to a stateful function or script returns back a State object that lets us consume the program from the outside. (This is similar to what generator functions do.)

Return Value

The State object features a value property that carries the program's actual return value:

function** sum(a, b) {
  return a + b;
}

const state = sum(5, 4);
console.log(state.value); // 9

But given a "live" program, the state.value property also comes as a "live" property that always reflects the program's new return value should anything make that change:

function** counter() {
  let count = 0
  setInterval(() => count++, 500);
  return count;
}

const state = counter();
console.log(state.value); // 0

Now, the general-purpose, object-observability API: Observer API puts those changes right in our hands:

Observer.observe(state, 'value', mutation => {
  //console.log(state.value); Or:
  console.log(mutation.value); // 1, 2, 3, 4, etc.
});

Module Exports

For module programs, the State object also features an exports property that exposes the module's exports:

// Stateful module
const program = new StatefulModule(`
  import module1, { module2 } from 'package-name';
  import { module3 as alias } from 'package-name';
  ...
  export * from 'package-name';
  export let localVar = 0;
`);

const state = await program.execute();
console.log(state.exports); // { module1, module2, module3, ..., localVar }

But given a "live" program, each property in the state.exports object also comes as a "live" property that always reflects an export's new value should anything make that change:

// As module
const program = new StatefulModule(`
  export let localVar = 0;
  ...
  setInterval(() => localVar++, 500);
`);

const state = await program.execute();
console.log(state.exports); // { localVar }

Now, again, the Observer API puts those changes right in our hands:

Observer.observe(state.exports, 'localVar', mutation => {
  //console.log(state.exports.localVar); Or:
  console.log(mutation.value); // 1, 2, 3, 4, etc.
});

// Observe "any" export
Observer.observe(state.exports, mutations => {
  mutations.forEach(mutation => console.log(mutation.key, mutation.value));
});

Disposing Stateful Programs

Stateful programs may maintain many live relationships and should be disposed when their work is done! The State object they return exposes a dispose() method that lets us do just that:

state.dispose();

Interaction with the Outside World

Stateful programs can read and write to the given scope in which they run; just in how a regular JavaScript function can reference outside variables and also make side effects:

let a = 2, b;
function** bar() {
  b = a * 2;
}
bar();

But unlike regular JavaScript, Stateful programs maintain a live relationship with the outside world:

...with Arbitrary Objects

With any given object, every interaction happening at the property level is potentially reactive! This means that:

Mutations to Object Properties from the Outside Will Be Automatically Reflected

Stateful JS programs will statically reflect changes to any property that they may depend on:

// External value
const foo = { baz: 0 };

function** bar() {
  let localVar = foo.baz;
  console.log(localVar);
}
bar();

whether it's a reference made from within program body itself as above, or from the place of a parameter's default value:

function** bar(localVar = foo.baz) {
  console.log(localVar);
}
bar();

This will now be reflected above:

// Update external dependency
foo.baz = 1;

In practice...

...since the Observer API isn't yet native, the above foo.baz = 1 assignment would need to happen via the Observer.set() method:

Observer.set(foo, 'baz', 1);

Interactions with Arbitrary Objects from the Inside Are Observable

Mutations from within a Stateful program may conversely be observed from the outside:

// External value
const foo = { baz: 0 };

// Observe specific property
Observer.observe(foo, 'baz', mutation => {
  console.log(mutation.type, mutation.key, mutation.value, mutation.oldValue);
});

The following operation will now be reported above:

function** bar() {
  foo.baz++;
}
bar();

And if you'd go further with the Observer API, you could even intercept every access to an object's properties ahead of Stateful programs!

Example

// Intercept specific property
Observer.intercept(foo, {
    get:(e, recieved, next) => {
        if (e.key === 'props') {
          return next(['prop1', 'prop2']);
        }
        return next();
    },
});

...with the Global Scope

For global variables, interactions happening directly at the variable level, not just at the property level this time, are potentially reactive! (Here we take advantage of the fact that global variables are actually properties of a real object - the globalThis - which serves as JavaScript's global scope!)

This means that:

Changes to the Global Scope from the Outside Will Be Automatically Reflected

Stateful JS programs will statically reflect changes to any global variable that they may depend on:

// External value
var baz = 0;
// Or: globalThis.baz = 0;

function** bar() {
  let localVar = baz;
  console.log(localVar);
}
bar();

whether it's a reference made from within program body itself as above, or from the place of a parameter's default value:

function** bar(localVar = baz) {
  console.log(localVar);
}
bar();

This will now be reflected above:

// Update external dependency
baz = 1;

In practice...

...since the Observer API isn't yet native, the above baz = 1 assignment would need to happen via the Observer.set() method:

Observer.set(globalThis, 'baz', 1);

Interactions with the Global Scope from the Inside Are Observable

Updates to global variables from within a Stateful program may conversely be observed from the outside:

// External value
var baz = 0;

// Observe specific variable
Observer.observe(globalThis, 'baz', mutation => {
  console.log(mutation.type, mutation.key, mutation.value, mutation.oldValue);
});

The following operation will now be reported above:

function** bar() {
  baz++;
}
bar();

And if you'd go further with the Observer API, you could even intercept every access to global variables ahead of Stateful programs!

Example

// Intercept specific property
Observer.intercept(globalThis, {
    get:(e, recieved, next) => {
        if (e.key === 'props') {
          return next(['prop1', 'prop2']);
        }
        return next();
    },
});

...with Stateful Parent Scopes Themselves

While bare variables in a local scope in JavaScript don't map to a physical, observable object like we have of global variables, bare variables in a Stateful scope are potentially reactive like we have of global variables.

Where a function runs within a Stateful program itself, any updates it makes to those variables are automatically reflected:

(function** () {
  // Stateful scope

  let count = 0;
  setInterval(() => count++, 500); // Live updates, even from within a non-stateful closure

  // "count" is automatically reflected here
  console.log('From main stateful scope: ', count);

  function** nested() {
    // "count" is automatically reflected here
    console.log('From inner stateful scope: ', count);
  }
  nested();

})();

Inside a Stateful Program (How It Works!)

In how Stateful programs can already entirely manage themselves, knowledge of how they work is very much optional! But, if you may look, this section covers just that very awesome part!

Knowing how things work presents a great way to reason about Stateful programs, and a better background for taking full advantage of the "Stateful" magic to never again do manual work!

• Reactivity
• Sensitivity
• Update Model
• Flow Control
• Experimental Features

Polyfill

Stateful JS may be used today via a polyfill. And good a thing, while this is a full-fledged compiler at heart, there is no compile step required, and you can have all of Stateful JS live in the browser!

Load from a CDN
└─────────

<script src="https://unpkg.com/@webqit/stateful-js/dist/main.js"></script>

└ This is to be placed early on in the document and should be a classic script without any defer or async directives!

// Destructure from the webqit namespace
const { StatefulFunction, StatefulAsyncFunction, StatefulScript, StatefulModule, State, Observer } = window.webqit;

Install from NPM
└─────────

// npm install
npm i @webqit/stateful-js

// Import API
import { StatefulFunction, StatefulAsyncFunction, StatefulScript, StatefulAsyncScript, StatefulModule, State, Observer } from '@webqit/stateful-js';

See details

API	Equivalent semantics...
StatefulFunction	function** () {}
StatefulAsyncFunction	async function** () {}
StatefulScript	<script>
StatefulAsyncScript	<script async>
StatefulModule	<script type="module">

While fully supporting program-level APIs - StatefulScript, StatefulAsyncScript, StatefulModule, the current polyfill only supports the constructor forms - StatefulFunction, StatefulAsyncFunction - of Stateful Functions - which give you the equivalent of the normal function forms!

Code

// External dependency
globalThis.externalVar = 10;

// StatefulFunction
const sum = StatefulFunction(`a`, `b`, `
  return a + b + externalVar;
`);
const state = sum(10, 10);

// Inspect
console.log(state.value); // 30
// Reflect and inspect again
Observer.set(globalThis, 'externalVar', 20);
console.log(state.value); // 40

But the double star syntax is supported from within a Stateful program itself:

Code

const program = StatefulFunction(`
  // External dependency
  let externalVar = 10;

  // StatefulFunction
  function** sum(a, b) {
    return a + b + externalVar;
  }
  const state = sum(10, 10);

  // Inspect
  console.log(state.value); // 30
  // Reflect and inspect again
  externalVar = 20;
  console.log(state.value); // 40
`);
program();

Stateful JS Lite

It is possible to use a lighter version of Stateful JS where you want something further feather weight for your initial application load. The Lite version initially comes without the compiler and yet lets you work with Stateful JS ahead of that.

Load from a CDN
└─────────

<script src="https://unpkg.com/@webqit/stateful-js/dist/main.async.js"></script>

└ This is to be placed early on in the document and should be a classic script without any defer or async directives!

// Destructure from the webqit namespace
const { StatefulAsyncFunction, StatefulAsyncScript, StatefulModule, State, Observer } = window.webqit;

Install from NPM
└─────────

// npm install
npm i @webqit/stateful-js

// Import Lite API
import { StatefulAsyncFunction, StatefulAsyncScript, StatefulModule, State, Observer } from '@webqit/stateful-js/async';

See details

API	Equivalent semantics...
StatefulAsyncFunction	async function** () {}
StatefulAsyncScript	<script async>
StatefulModule	<script type="module">

Here, only the "async" program types can possibly be obtained this way!

Code

// External dependency
globalThis.externalVar = 10;

// StatefulFunction
const sum = StatefulAsyncFunction(`a`, `b`, `
  return a + b + externalVar;
`);
const state = await sum(10, 10);

// Inspect
console.log(state.value); // 30
// Reflect and inspect again
Observer.set(globalThis, 'externalVar', 20);
console.log(state.value); // 40

Good a thing, these specific APIs take advantage of the fact that they can do compilation for their program types off the main thread! Thus, as a perk, the compiler is loaded into a Web Worker and all compilations happen off the main thread!

But having been designed as a movable peice, the Stateful JS Compiler is all still loadable directly - as if short-circuiting the lazy-loading strategy of the Lite APIs:

<head>
 <script src="https://unpkg.com/@webqit/stateful-js/dist/compiler.js"></script> <!-- Must come before the polyfil -->
  <script src="https://unpkg.com/@webqit/stateful-js/dist/main.async.js"></script>
</head>

Examples

Using the Stateful JS and Observer API polyfills, the following examples work today.

• Example 1: Reactive Custom Elements
• Example 2: Pure Computations

Example 1: Reactive Custom Elements

Manual reactivity accounts for a large part of the UI code we write today. But, what if we could simply write "Stateful" logic?

In this example, we demonstrate a custom element that has a Stateful render() method. We invoke the render() method only once and let every subsequent prop change be statically reflected:

Code

customElements.define('click-counter', class extends HTMLElement {

  count = 10;

  connectedCallback() {
    // Initial rendering
    this._state = this.render();
    // Static reflect at click time
    this.addEventListener('click', () => {
      this.count++;
      //Observer.set(this, 'count', this.count + 1);
    });
  }

  disconnectCallback() {
    // Cleanup
    this._state.dispose();
  }

  // Using the StatefulFunction constructor
  render = StatefulFunction(`
    let countElement = this.querySelector( '#count' );
    countElement.innerHTML = this.count;
    
    let doubleCount = this.count * 2;
    let doubleCountElement = this.querySelector( '#double-count' );
    doubleCountElement.innerHTML = doubleCount;
    
    let quadCount = doubleCount * 2;
    let quadCountElement = this.querySelector( '#quad-count' );
    quadCountElement.innerHTML = quadCount;
  `);

});

Example 2: Pure Computations

Even outside of UI code, we often still need to write reactive logic! Now, what if we could simply write "Stateful" logic?

In this example, we demonstrate a simple way to implement something like the URL API - where you have many interdependent properties!

Code

class MyURL {

  constructor(href) {
    // The raw url
    this.href = href;
    // Initial computations
    this.compute();
  }

  compute = StatefulFunction(`
    // These will be re-computed from this.href always
    let [ protocol, hostname, port, pathname, search, hash ] = new URL(this.href);

    this.protocol = protocol;
    this.hostname = hostname;
    this.port = port;
    this.pathname = pathname;
    this.search = search;
    this.hash = hash;

    // These individual property assignments each depend on the previous 
    this.host = this.hostname + (this.port ? ':' + this.port : '');
    this.origin = this.protocol + '//' + this.host;
    let href = this.origin + this.pathname + this.search + this.hash;
    if (href !== this.href) { // Prevent unnecessary update
      this.href = href;
    }
  `);

}

└ Instantiate MyURL:

const url = new MyURL('https://www.example.com/path');

└ Change a property and have it's dependents auto-update:

url.protocol = 'http:'; //Observer.set(url, 'protocol', 'http:');
console.log(url.href); // http://www.example.com/path

url.hostname = 'foo.dev'; //Observer.set(url, 'hostname', 'foo.dev');
console.log(url.href); // http://foo.dev/path

Relationship with Other Concepts

TODO

Getting Involved

All forms of contributions are welcome at this time. For example, syntax and other implementation details are all up for discussion. Also, help is needed with more formal documentation. And here are specific links:

• Project
• Documentation
• Discusions
• Issues

License

MIT.