chifley

🚂 chifley

🚨🚨                                                                    🚨🚨
🚨 Warning, this library is very new and should be considered pre-release. 🚨
🚨                         Do not use in production.                       🚨
🚨🚨                                                                    🚨🚨

A small, speedy reactive programming library with a simple API designed to be app ready.

• 👶 Simple reactivity model
• 🤖 Easy to debug
• 📉 Low memory use
• 🏃‍♂️‍➡️ Fast
• 🤏 Tiny (8kb) but with just enough features ⭐

Quick Start

import { Stream as c } from 'chifley'

const text = c('wow')

// perfectly valid email validation 🫠
const isEmail = text.map( text => text.includes('@') )

isEmail.get()
//=> false

try {
    isEmail.set(true)
} catch (e) {
    // Cannot write to a readonly stream!
    console.error(e) 
}

text.set('[e-mail]')

isEmail.get()
//=> true

Why Chifley?

What makes Chifley different from other stream libraries? Chifley is focused on one goal: the perfect stream libraries for building UIs. In order to meet that goal, Chifley also has to be fast, easy to use (and debug!) and it has to be small.

You'll find Chifley familiar if you've used libraries like flyd or mithril-stream. If you've never used a stream library before then Chifley is a great place to start, it is designed to be easy to learn. If you've coming from libraries like Rx.js you need to unlearn what you have learned as Chifley has a very different philosophy and semantics.

API

Creating a stream

You can initialize a stream with an initial value or without. If you opt of an initial value the type of the stream will be T | undefined otherwise it will be T.

import { Stream as c } from 'chifley'

const name = c('Ringo')
const songs = c<{ track: string, duration: number }>()

Reading stream values

The last emitted value of a stream is always cached and can be retrieved via .get()

const name = c('Ringo')

c.get()
//=> 'Ringo'

If the stream had no initial value, the type will be T | undefined:

const name = c<string>();

c.get().toUpperCase() 
// 🚫 Not OK: Typescript Error 

c.get()?.toUpperCase() 
// 🟢 OK

Writing stream values

You can change a stream value via .set

name.set('George')

This will trigger dependent streams to also update.

If you need to access the previous value as part of the update you can use .update

name.update( prev => prev == 'John' ? 'Paul' : 'George' )

Subscribing to changes

You can subscribe to changes via .map. It will only emit when it has received a value. If you don't initialize a stream, don't worry, it won't emit with undefined.

const name = c<string>();

name.map( x => console.log('The name changed: ', x))

// At this point nothing has logged.

name.set('Paul')
// Logs: 'The name changed: Paul'

Ending a stream

You can end a stream and all its dependent streams by calling .end(). You can also check if a stream has ended via .end.state === State.Ended

name.end()

Opting out of an update

Within a visitor function you can return c.SKIP which will be detected and skip propagation of that stream and its dependencies. The existing value of the stream will be retained and no dependencies will be updated.

const year = c(2000)

const olympicYear = age.map( x => x % 4 === 0 ? x : c.SKIP )

// immediately logs: 'olympics! 2000'
olympicYear.map( x => {
    console.log('olympics!', x)
})

year(1999)
// no log occurs

year(2024)
// logs: 'olympics! 2024'

This allows .map (and most other operators) to act like .filter or .reject which can be incredibly powerful.

🤓 Note unlike mithril-stream SKIP only works as a return value within a stream transform. You can write SKIP to a writable stream but it will be treated just like any other value. This removes two logical branches in the update path and speeds things up enough to warrant the sacrifice.

Static methods and properties

merge

If you have two or more streams and you would like to combine them in some way then merge is the way to go.

You provide a list of streams and it will provide a stream of a list of values.

const organization_id = c<string>()
const project_id = c<string>()

const ids = c.merge([organization_id, project_id] as const)

const path = 
    ids.map(([organization_id, project_id]) => `/org/${organizaton_id}/project/${project_id}`)

path.map( 
    path => console.log('URL path changed: ', path)
)

merge will wait for all streams to have at least 1 value/emit before it emits its first update.

// So far nothing has logged
organization_id.set('1')

// still nothing has logged

project_id.set('2')
// now this logs:
//  'URL path changed: /org/1/project/2

• isStream

Not very exciting but if you want to know if some arbitrary object is an instance from this library use this:

isStream(null) 
//=> false

isStream( c('hello') )
//=> true

Instance operators

map

When you have one input stream and you want to transform it and produce a new output stream use .map

const name = c('George')

const isCoolBeatle = name.map( name => name === 'Ringo' )

If you just want to subscribe to a stream and not produce a new stream, just use .map, it is okay, it is no more or less expensive and the FP wizard cabal will let you off, just this once.

dropRepeats / dropRepeatsWith

Do you have a stream that emits all the time, like say when a user does stuff? Do you want to only be notified when we have a new value? Then use dropRepeats

const food = c<string>()

const foodOrder = food.dropRepeats()

foodOrder.map( x => console.log('Hurry up!  I want my ', x + '!'))

food.set('pretzel') // Hurry up!  I want my pretzel!
food.set('pretzel') 
food.set('pretzel')
food.set('pretzel')

food.set('doughnut') // Hurry up!  I want my doughnut!

dropRepeats using reference equality, if you'd like to change the definition of equality, use dropRepeatsWith( ... ) with a custom equality function e.g. for deep equality you could use Ramda's R.equal

const stateChanges = state.dropRepeatsWith(R.equal)

filter / reject

For opting in and out of changes we can use c.SKIP but sometimes defining control flow with simple predicates is a lot clearer.

For these casesfilter and reject can help clean up things a bit.

E.g. only eit a terminal if the line feed exactly equals 'exit':

const linefeed = c<string>()

const exit = linefeed.filter( x => x == 'exit' )

exit.map( () => killTerminalSubProcess() )

Or only execute statements that don't start with a comment:

const queryInput = c<string>()

const notAComment = linefeed.reject( x => x.startsWith('--') )

const completedStatement = notAComment.filter( x => x.includes(';') )

completedStatement.map( query => {
    execSqlQuery(query)
})

default

If you want to guarantee a stream has a value you can use default. Note this isn't used for coalescing undefined or null values, this is for handling streams that may not have emitted a value yet.

const projects = c<Project[]>()

{
    // 🚫 Type error and will result in an `undefined` `.length `value
    const length = projects.map( x => x.length )
}

{
    // 🟢 OK
    const length = projects.default([]).map( x => x.length )
}

skip

Sometimes we have a stream with an initial value but we don't want to be notified until we receive a new value. This is when we use skip:

const form = c<InitialState>(init())

const changed = form.skip(1)

changed.map( x => console.log('Form changed', x))

take / once

If we want to only get the first emit we can use .once().

const firstClick = click.once()

If you want the first n emits you can use .take(n)

const firstThreeClicks = click.take(3)

In both cases the stream will end once it reaches the maximum number of events.

scan

scan makes it convenient to produce a new value based on the combination of the latest and previous emit. scan is often used as a lightweight alternative to redux or for building reactive state machines.

Here is a simple counter example:

type Action = 'inc' | 'dec'
const action = c<Action>()

const count = action.scan(0, (prev, action) => {
    if (action === 'inc' ) {
        return prev + 1
    } else if (action == 'dec') {
        return prev - 1
    }
    return prev
})

awaitLatest

Unlike other stream/obserable libraries Chifley doesn't coerce promises, or arrays (or anything else) into stream events. A Promise is as much a value as a number of a string.

But we do provide some basic operators for handling the two common cases of coercing promises into stream events.

awaitLatest allows you to preference the promise resolution of the latest emit even if a prior promise resolves first. This is most useful for network requests:

const results = 
    searchInput
        .awaitLatest((search) = fetch(`${endpoint}?q=${search}`).then( x => x.json()))

search('hello')

results.map( x => {
    console.log(...x.status === 'fulfilled' ? ['🟢', x.value] : ['🚫', x.reason])
})

In the above example, the user may constantly be updating the value of searchInput but we are guaranteed to only get the network response corresponding to the latest searchInput.

Because Promise's can reject we coerce the result to the allSettled API

type Settled = 
    | { status: 'fulfilled', value: T }
    | { status: 'rejected', reason: unknown }

Note this will help us with our control flow but it won't actually cancel the network request, we'll simply ignore the response of older requests.

For convenience though, awaitLatest creates an AbortController and passes the signal in as a second argument so you can optionally abort requests (or anything else that adheres to the AbortController API) when a new request is starting.

const results = 
    searchInput
        .awaitLatest((search, { signal }) = fetch(`${endpoint}?q=${search}`, { signal }).then( x => x.json()))

awaitEvery

Sometimes though we want a corresponding emit for every single promise resolution

const results = 
    searchInput
        .awaitEvery((search, { signal }) = fetch(`${endpoint}?q=${search}`, { signal }).then( x => x.json()))

With awaitEvery for every input we will get a corresponding emit (providing the promise settles). We still provide the ability to cancel on abort signal but this will only fire if the stream is ended before a request can resolve.

Like awaitLatest, the resulting emit is of the Settled<T> type.

throttle

Usually we don't want to fire a network request for every single corresponding user keystroke.

One strategy is to throttle the input, sending at most 1 request every n milliseconds.

const searchInput = c('')

searchInput
    .throttle(100)
    .awaitLatest((search, { signal }) = fetch(`${endpoint}?q=${search}`, { signal }))

afterSilence

Other times we only want to fire off a request when the user stops typing for a bit.

const searchInput = c('')

searchInput
    .afterSilence(100)
    .awaitLatest((search, { signal }) = fetch(`${endpoint}?q=${search}`, { signal }))

Advanced

Error / Promise Rejection propagation

This library deliberately has no opinions on synchronous error handling. If you throw an error within a stream update it will simply bubble up the stack uninterrupted. However, for awaitLatest and awaitEvery we return the same type as the standard Promise.allSettled API.

Stream End Behaviour

When you end a stream the last emitted value will still be cached so .get() will continue to work forever. But any upstream changes will not propagate to the ended stream, the value will stay the same, no operators will fire.

All children streams will also be ended (recursively).

If you end a writable stream and change the value via .set or .update no exception will be thrown and the value will change. If you subscribe to an ended stream via an operator no exceptions will be thrown but as the upstream dependency will never again emit, neither will the new dependent stream.

When using .merge, if any dependency ends the merged stream is ended. If another (unended) dependency emits, the merged stream won't emit.

Tracking references and creations

This feature is designed to be used primarily by framework authors and not directly by Chifley users in normal application code. It will seem a little verbose at first but it is designed to give the framework author precise control over how tracking contexts cascade.

To track stream references use the static trackReferenced function:

const returnValue = c.trackReferenced(() => {
    return someFunctionThatMightReferenceStreams()
}, new Set<c>)

Any synchronous calls to .get() while that function runs will result in that stream being added to the provided set Set<c>. You can then inspect the set after executing the function and use it to inject dependencies for some framework process, e.g. automatic rendering when a stream emits.

To track stream creation, you can do the same thing via trackCreated:

const returnValue = c.trackCreated(() => {
    return someFunctionThatMightCreateStreams()
}, new Set<c>)

This is useful if you'd like to automatically destroy streams created within a particular scope, such as during a render, within a component initialization etc.

To track both creations and references use c.track

const returnValue = c.track(() => {
    return someFunctionThatMightCreateStreams()
}, new Set<c>)

You can opt out of a tracking context using the sample function. There are sampleCreated and sampleReferenced variants (which do what you would expect).

const returnValue = c.track(() => {
    someFunctionThatMightCreateStreams()

    return sample(() => {
        return weWillNotTrackThisFunctionCall()
    })
}, new Set<c>)

You can also untrack a stream that has already been tracked, this just removes it from whatever tracking context is currently active.

const returnValue = c.track(() => {
    someFunctionThatMightCreateStreams()

    // even if `someFunctionThatMightCreateStreams` referenced 
    // `someStream` it won't appear in the set.
    untrack(someStream);
}, new Set<c>)

From this feature set you can build S.js like computations, and any kind of conceivable reactive framework feature you'd like to build. If you'd like tracking contexts to cascade, just reuse the same set for multiple contexts. If you'd like each tracking context to have sole responsibility then use separate sets. If you'd like to use generator functions then you can start and end a tracking set for each invocation of iterator.next(...) - the sky is the limit.

Comparisons

👶 Simple reactivity model

Chifley is heavily inspired by stream libraries like flyd and mithril-stream. All Chifley streams are immediately active. What does "active" mean? It means there is no extra subscribe() call you have to do to have your stream transforms start evaluating. If you put a console.log(...) in a call to filter or map or any other operator, you'll see they are immediately invoked.

Additionally, every stream caches the last emitted value, and you can always access it via .get(). Source streams can be written to via .set and .update while dependent streams are readonly and will prevent you from writing to them both at runtime and via Typescript.

When you end a parent stream all dependent streams will also end.

If you are coming from libraries like Rx.js this may feel a little foreign as you don't typically manually end a source stream you simply unsubscribe from a sink stream and the source is automatically ended when there's no more subscribers. But for UI programming specifically explicitly ending source streams is both more efficient and easier to reason about.

Unlike traditional FRP libraries you are encouraged to write to streams (even while changes are already propagating). You are expected to guard against potential infinite loops yourself, but Chifley is designed to make it easy to diagnose issues in your stream graph by using a transaction / atomic clock model: each new write is deferred until the immediate effects of the current write are complete.

🤖 Easy to debug

Most stream libraries have a complex recursive propagation algorithm. If you are trying to debug your stream graph, traditional debuggers aren't very helpful as you will find yourself stepping through generic internal functions recursively.

So, instead, we find ourselves using custom tools like Rx marbles (and more commonly, just logs 🫠).

Chifley was deliberately designed to be sustainable at scale with large complex UI stream graphs. When you update a stream, Chifley traverses all the possible dependencies in a single while loop.

Additionally, any writes that occur during a propagation are deferred until the immediate impact of the previous write is complete.

This doesn't prevent infinite loops but it makes it very clear when you are entering and exiting a propagation caused by a single write. It also helps identify how many dependent transactions are spawned by a single write.

📉 Low memory use

Chifley streams are instances of a simple class. Each instance has many useful methods available but thanks to prototypical inheritance, the memory for those methods are all shared. Each stream has a few instance specific properties but everything else is shared and stored once.

🏃‍♂️‍➡️ Fast

Chifley isn't the fastest stream library available but it is competitive and strikes a healthy balance between memory / CPU usage and features.

• From Github Actions

• Benchmarks Source

🤓 Note these bench marks are very early and incomplete. Contributions / extensions would be very welcome.

🤏 Tiny but with just enough features ⭐

Chifley originally started as a fork of mithril-stream, which in turn was a rewrite of flyd. The code has evolved greatly over many years, and this library is a formalization of a lot of things we have learnt over nearly a decade of working with these libraries.

We observed what methods and capabilities weer actually used, and we have removed everything we didn't. We even removed things we really liked in principle.

The truth is, when your stream library uses the flyd model, you don't need as many operators or abstractions as you do in say Rx.js because we aren't trying to prevent side effects by predefining operators for all possible scenarios.

So here's a short list of things we have included in this tiny library:

• 💪 14 commonly use stream operations: (map, filter, reject, merge, scan, dropRepeats, throttle, default, skip, take, once, afterSilence, awaitLatest and awaitEvery)
• 🕵️ Automatic dependency tracking (like S.js / Signal libraries)
• 🔒 Read-only streams (both at runtime and via Typescript)
• 💾 toJSON serialization
• 🔥 Sin.js observable protocol
• 🕰️ Predictable atomic clock update (similar to S.js)

Traditional FRP vs Reactive Store

There are many stream libraries in JS, with varying philosophies, feature sets and performance profiles.

Chifley is lazer focused on UI state management, nothing more. It is designed to act as a simple reactive store with a few core transforms/combinators. You are encouraged to write to a chifley stream the same way you write to setState in React, or a signal in Solid.js.

But in traditional FRP this is considered problematic. You will often read advice like:

If you find yourself writing to a Subject you are probably doing things the wrong way

And in some contexts this is true, you really can create a mess very quickly if you don't take the time to learn how to use these tools in the way they were designed to be used.

But part of the benefit of Chifley is you don't need to explicitly think about hot vs cold, MemoryStreams, share/publish etc. It's effectively a state store with a few tricks up its sleeve.

Apples to Apples comparisons are complicated...

Often with traditional FRP libraries you can really tweak performance if you know what you are doing but its also easy to get awful performance if you are not very careful. It is quite simple to find yourself with duplicated effects (e.g. network requests) because multiple components are referencing the same observable and each subscription duplicates all the transforms.

This will never happen with libraries where all streams are "hot" (xstream, flyd, mithril-stream) but can be avoided with specific operators in "cold" by default libraries (Rx.js, most.js).

xstream is a very interesting case, it has amazing performance and while being Rx inspired it makes some important design decisions to side step alot of the confusion. xstream is a great choice if you are writing mostly acyclic stream graphs and you are happy to avoid writing to subject/source streams. It is still very much in the spirit of what most would consider "real FRP". In my benchmarks (which are very tailored towards UI workloads) xstream was frequently the fastest stream library that I tested against. xstream is roughy 2x faster than Chifley.

flyd and mithril-stream both could be considered direct ancestors of Chifley. The programming model is less focused on linear dataflow and raw performance and instead is designed to assist with UI programming. Chifley is roughly 4x faster than flyd, and 18x faster than mithril-stream.

Chifley approaches (and sometimes exceeds) the performance of [most.js] but only in the specific context of how we benchmark (multiple subscribers, no fromArray, no special caching operators). In a different context [most.js] is often the leader in performance, especially in server workloads where there is 1 subscriber per request. For those specific use cases I would recommend to look at [most.js]

Benchmarking considerations

In our benchmarks we completely avoid fromArray and any kind of operator like share(), remember() or publish(). We avoid fromArray because this is easy to optimize with specialized code paths but its not a good representation of performance when processing asynchronous events. We avoid operators like share(), remember(), publish() (etc) because having to opt in to this specialized code path requires a level of expertise that isn't representative of most users of a stream library. We want to know how Chifley compares to the sort of code you would actually see in the wild, equally Chifley benchmarks are not allowed to use any special operator to improve performance, if we're filtering, mapping and reducing, then that's all the operators we can use in the benchmark.

Additionally, its rare in a UI for a stream to have only a single subscriber, so we will often subscribe to the same pipeline n times to see how performance degrades (this heavily impacts benchmarking scores for cold observable libraries).

 filter -> map -> reduce 1000000 integers (10 consumers)
-------------------------------------------------------
xstream              34.83 op/s ±  1.89%   (82 samples)
chifley              21.05 op/s ±  7.83%   (53 samples)
flyd                  4.70 op/s ±  1.26%   (27 samples)
mithril-stream        1.76 op/s ±  3.62%   (13 samples)
most.js               8.46 op/s ±  8.69%   (44 samples)
-------------------------------------------------------

From Github Actions

Prior Art and Thanks

This library has a lot of influences and is greatly shaped by years of working with FP and FRP libraries in large complex applications.

Many thanks first of all to Simon Friis Vindum, this library is greatly influenced by his library flyd.

Much appreciaton to Adam Haile for creating S.js and to Ryan Carniato for popularising it through his work with Solid.js. This library has been inspired by S.js in its atomic update algorithm and with the automatic dependency tracking that we also see in S.computation.

After benchmarking, we greatly revised the internals to mimic some techniques xstream uses to improve performance. A lot of credit goes to André Staltz, Tyler Steinberger and all the other contributors to xstream for making it so fast.

This library originally started as a port of mithril-stream, and mithril-stream was originally a rewrite of flyd. mithril-stream was first implemented by Leo Horie (Original author of Mithril.js) and later rewritten by Rasmus Porsager. Thanks again goes to Rasmus for the Sin.js observable protocol which this library has adopted.

Finally, special thanks to Barney Carroll for constantly offering helpful feedback and reading early drafts of anything and everything I ever work on.