[!IMPORTANT]
‼️ Announcing the thi.ng user survey 2024 📋
Please participate in the survey here!
(open until end of February)
To achieve a better sample size, I'd highly appreciate if you could
circulate the link to this survey in your own networks.
Discussion
[!NOTE]
This is one of 189 standalone projects, maintained as part
of the @thi.ng/umbrella monorepo
and anti-framework.
🚀 Help me to work full-time on these projects by sponsoring me on
GitHub. Thank you! ❤️
About
Clojure inspired, composable ES6 iterators & generators.
Most of the provided functionality here is also available in the form of
more composable & efficient transducers via
@thi.ng/transducers.
Status
DEPRECATED - superseded by other package(s)
Search or submit any issues for this package
Related packages
Installation
yarn add @thi.ng/iterators
ES module import:
<script type="module" src="https://cdn.skypack.dev/@thi.ng/iterators"></script>
Skypack documentation
For Node.js REPL:
const iterators = await import("@thi.ng/iterators");
Package sizes (brotli'd, pre-treeshake): ESM: 2.46 KB
Dependencies
API
Generated API docs
In alphabetical order:
butLast
Signature: butLast<T>(input: Iterable<T>) => IterableIterator<T>
Yields iterator of all but the last value of input.
[...ti.butLast([])]
[...ti.butLast([1])]
[...ti.butLast([1,2])]
[...ti.butLast([1,2,3])]
[...ti.butLast("hello")]
[...ti.butLast(ti.range(10))]
cached
Signature: cached<T>(input: Iterable<T>) => CachedIterableIterator<T>
Consumes and lazily caches values of a finite input and returns
a no-arg function, which when called return new iterator. These iterator
instances always start from the beginning of the cache and allows for
separate states and sharing of cached results among arbitrary number of
iterators. The original input is only consumed when attempting to read
beyond current cache boundary.
c = ti.cached(ti.range(10));
a = c();
b = c();
a.next();
b.next();
b.next();
a.next();
angles = ti.cached(ti.range(0, 360, 45));
[...angles()]
ti.zip(
angles(),
ti.map(
ti.juxt(Math.sin, Math.cos),
ti.map(
(x)=> x * 180 / Math.PI,
angles()
)
)
);
concat
Signature: concat<T>(...inputs: Iterable<T>[]) => IterableIterator<T>
Produces iterator yielding lazy concatenation of given iterables.
For practical purposes none but the last input should be infinite. Any
null
or undefined
input arguments are skipped in the output.
[... ti.concat([1, 2, 3], [10, 20, 30], [100, 200, 300])]
[...ti.concat.apply(null, ["abc", null, [1, 2, 3]])]
constantly
Signature: constantly<T>(x: T) => (...args: any[]) => T
Helper function returning a new fn which takes any number of args and
always returns x
.
iter = ti.takeWhile(x => x < 0.98, ti.repeatedly(()=> Math.random()));
ti.reduce((a, b)=> a + b, 0, ti.map(ti.constantly(1), iter))
times10 = (flt, coll) => [...ti.map(x => x * 10, ti.filter(flt, coll))];
filterModes = {
odd: x => (x % 2) == 1,
even: x => (x % 2) == 0,
all: ti.constantly(true)
};
times10(filterModes[???], [1, 2, 3]);
consume
Signature: consume(input: Iterator<any>, n?: number) => void
Helper function. Consumes & discards items from iterator (possibly for
side effects) and optionally only up to the given number of steps.
cycle
Signature: cycle<T>(input: Iterable<T>) => IterableIterator<T>
Produces iterator which lazily caches & infinitely repeats
sequence of input. Important: Input MUST be finite, use take
to
truncate input or output if necessary.
[... ti.take(10, ti.cycle(ti.range(3)))]
dedupe
Signature: dedupe<T>(input: Iterable<T>) => IterableIterator<T>
Produces iterator which discards successive duplicate values from input.
Important: Uses ===
for equality checks.
[... ti.dedupe([1, 2, 2, 3, 4, 4, 4, 3])]
dedupeWith
Signature: dedupeWith<T>(equiv: (a:T, b: T) => boolean, input: Iterable<T>) => IterableIterator<T>
Like dedupe
, but uses given function equiv
to determine equivalence
of successive values.
var coll = [{ a: 1 }, { a: 1, b: 2 }, { a: 2, b: 2 }, { a: 2, b: 2 }, { a: 3 }];
var eq = (a, b) => a.a === b.a;
[...ti.dedupeWith(eq, coll)]
dense
Signature: dense<T>(input: Iterable<T>) => IterableIterator<T>
Yields iterator of all non-null
and non-undefined
values of input
(e.g. a sparse array).
var a = []
a[10] = 1;
a[20] = 2;
[...ti.dense(a)]
drop
Signature: drop<T>(n: number, input: Iterable<T>) => IterableIterator<T>
Consumes & discards up to n
items from input and returns remaining
(possibly exhausted) iterator.
[... ti.drop(5, ti.range(10))]
[... ti.drop(5, ti.range(3))]
[... ti.take(5, ti.drop(5, ti.range()))]
dropNth
Signature: dropNth<T>(n: number, input: Iterable<T>) => IterableIterator<T>
Produces iterator which drops every n
th item from input.
[... ti.dropNth(2, ti.range(10))]
[... ti.dropNth(3, ti.range(10))]
dropWhile
Signature: dropWhile<T>(pred: (x: T) => boolean, input: Iterable<T>) => IterableIterator<T>
Consumes input and calls pred
for each item. Discards all items whilst
pred
returns true
, then returns remaining (possibly exhausted)
iterator.
[... ti.dropWhile((x) => x < 5, ti.range(10))]
every
Signature: every<T>(pred: (x: T) => boolean, input: Iterable<T>) => boolean
Consumes input and calls pred
for each item. When pred
returns not
true
, process stops and returns false
itself. When all items pass
the predicate, the function returns true
.
If input is empty/exhausted prior to execution, every
will return
false
.
var nums = ti.iterator([2, 4, 6, 8, 10]);
ti.every((x) => (x % 2) === 0, nums);
nums.next()
nums = ti.iterator([2, 3, 4]);
ti.every((x) => (x % 2) === 0, nums);
nums.next()
ti.every((x) => true, [])
filter
Signature: filter<T>(pred: (x: T) => boolean, input: Iterable<T>) => IterableIterator<T>
Consumes input and calls pred
for each item. Yields iterator of items
for which pred
returned true
.
var multOf3 = (x) => (x % 3) === 0;
[... ti.filter(multOf3, ti.range(10))];
flatten
Signature: flatten(input: Iterable<any>, flatObjects = true) => IterableIterator<any>
Produces iterator which recursively flattens input (using
flattenWith
). Important: Recursion only applies to iterable types
(excluding strings) and objects (enabled by default, using
objectIterator
, see below).
[... ti.flatten([1, [2, 3], [4, [5, ["abc"]]]])]
[...ti.flatten([{ a: 1 }, { a: 23, b: 42, c: [1, 2, 3] }])]
[...ti.flatten([{ a: 1 }, [1, 2, 3], { a: 23, b: 42, c: [1, 2, 3] }], false)]
flattenWith
Signature: flattenWith(tx: (x: any) => any, input: Iterable<any>) => IterableIterator<any>
Produces iterator which selectively & recursively flattens input. The
first arg tx
is a transformation fn called for each
non-null/undefined
value taken from the input. It's used to check and
possibly obtain an iteratable value for further flattening. The
transformer must return undefined
if the value can't or shouldn't be
flattened. If a value is returned it MUST be iterable.
The default transformer used by flatten
is:
let defaultTx = x =>
(typeof x !== "string" && (maybeIterator(x) || maybeObjectIterator(x))) ||
undefined;
let tx = x => typeof x == "string" ? ti.map(x => x.charCodeAt(0), x) : ti.maybeIterator(x);
[...ti.flattenWith(tx, ["ROOT", undefined, ["CHILD_1", null, ["CHILD_2"]]])]
fnil
Signature: fnil(fn: (...args: any[]) => any, ...ctors: (() => any)[]) => (...args: any[]) => any
Takes a function fn
and up to 3 ctor
functions. Produces a new
function that calls fn
, replacing a null
or undefined
arg with the
value obtained by calling its related positional ctor
fn (e.g. the
first ctor is used to supply first arg, etc.).
The function fn
can take any number of arguments, however only the
first 3 are being potentially patched, how many depends on the number of
ctor
fns supplied.
hello = (greet, name) => `${greet}, ${name}!`;
helloEN = ti.fnil(hello, () => "Hi", () => "user");
helloDE = ti.fnil(hello, () => "Hallo", () => `Teilnehmer #${(Math.random()*100)|0}`);
helloEN();
helleEN(null,"toxi");
helloEN("Howdy","toxi");
helloDE();
inc = x => x + 1
adder = ti.fnil(inc, () => 0);
adder();
adder(null);
adder(10);
updateWith = f => (obj, id) => { return obj[id] = f(obj[id]), obj; }
ti.reduce(updateWith(adder), {}, "abracadabra");
fork
Signature: fork<T>(input: Iterable<T>, cacheLimit = 16) => () => IterableIterator<T>
Similar to cached
, this function allows multiple consumers of a single
input, however is only using a sliding window of N cached values
(cached
stores the entire input).
fork
returns a no-arg function, which returns a new forked iterator
when called. There's no limit to the number of active forks.
Important: The use case for fork
is synchronized consumption at
similar speeds (up to cacheLimit
divergence). The cache is shared by
all forks. If one of the forks consumes the input faster than the
given cacheLimit
, the other forks will lose intermediate values. If in
doubt, increase the cache limit to a higher value (default 16). The
cache uses
@thi.ng/dcons
to avoid unnecessary copying during window sliding.
src = ti.partition(5, 1, ti.repeatedly(()=> (Math.random() * 100) | 0, 10));
f = ti.fork(src, 1);
raw = f();
sma = ti.map((part)=> ti.reduce((a, b) => a + b, 0, part) / part.length, f());
min = ti.map((part)=> ti.reduce((a, b) => Math.min(a, b), 100, part), f());
max = ti.map((part)=> ti.reduce((a, b) => Math.max(a, b), -1, part), f());
for(let part of raw) {
console.log(`part: ${part}, avg: ${sma.next().value}, min: ${min.next().value}, ${max.next().value}`);
}
frequencies
Signature: frequencies<T>(input: Iterable<T>, key?: (v: T) => any): IterableIterator<FrequencyPair<T>[]>
Consumes input, applies key
fn (if given) to each item and yields
iterator of 2-element tuples, each [key, freq]
(where freq
is the
number of times the item occurred). Important: The input MUST be
finite. Implementation uses JSON.stringify
to determine key equality.
If no key
fn is given, the original values will be used as key.
[... ti.frequencies([[1,2], [2,3], [1,2], [2,4]])]
[... ti.frequencies([1, 2, 3, 4, 5, 9, 3], (x) => x & ~1)]
var isLetter = (x) => /[a-z]/i.test(x);
[... ti.frequencies(ti.filter(isLetter, "hello world!"))].sort((a, b) => b[1] - a[1])
groupBy
Signature: groupBy<T>(key: (v) => any, input: Iterable<T>) => { [id: string]: T[] }
Consumes input, applies key
fn to each item and returns object of
items grouped by result of key
fn. Important: The input MUST be
finite. Implementation uses JSON.stringify
to determine key equality.
groupBy((x) => x & ~1, [1, 2, 3, 4, 5, 9, 3])
groupBy((x) => x.toUpperCase(), "AbRaCadaBra")
identity
Signature: identity<T>(x: T) => T
Helper function. Simply returns its argument.
ti.identity()
ti.identity(null)
ti.identity(42)
tests = [true, true, undefined, true]
ti.every(ti.identity, tests);
ti.every(ti.fnil(ti.identity, () => true), tests);
indexed
Signature: indexed<T>(input: Iterable<T>) => IterableIterator<[number, T]>
Yields iterator producing [index, value]
pairs of input.
[...ti.indexed("hello")]
interleave
Signature: interleave(...inputs: Iterable<any>[]) => IterableIterator<any>
Takes an arbitrary number of iterators and yields iterator of
interleaved items from each input. Terminates as soon as one of the
inputs is exhausted.
[... ti.interleave(ti.range(), ti.range(100, 200), ti.range(200, 205))]
interpose
Signature: interpose(x: any, input: Iterable<any>) => IterableIterator<any>
Produces an iterator which injects x
between each item from input
iter
.
[... ti.interpose("/", ti.range(5))]
iterate
Signature: iterate<T>(fn: (x: T) => T, seed: T) => IterableIterator<T>
Produces an iterator of the infinite results of iteratively applying
fn
to seed
.
Important: Use take
to truncate sequence.
[... ti.take(10, ti.iterate((x) => x * 2, 1))]
iterator
Signature: iterator<T>(x: Iterable<T>) => Iterator<T>
Syntax sugar. Returns x[Symbol.iterator]()
. Most functions in this
module call this automatically for each input.
juxt
Signature: juxt<T>(...fns: ((x: T) => any)[]) => (x: T) => any[]
Takes arbitrary number of functions and returns new function, which
takes single argument x
and produces array of result values of
applying each input function to x
(juxtoposition of the given
transformation functions).
var kernel = ti.juxt(
(x) => x - 1,
(x) => x,
(x) => x + 1
);
kernel(1)
[... ti.map(kernel, ti.range(3))]
last
Signature: last<T>(input: Iterable<T>) => T
Consumes a finite iterator and returns last item.
Important: Never attempt to use with an infinite input!
ti.last(ti.range(10))
ti.last(ti.take(10, ti.range()))
map
Signature: map<T>(fn: (...args: any[]) => T, ...inputs: Iterable<any>[]) => IterableIterator<T>
Consumes an arbitrary number of iterators and applies fn
to each of
their values. Produces iterator of results. Iteration stops as soon as
one of the inputs is exhausted. The mapping fn
should accept as many
arguments as there are inputs to map
. Provides a fast path for single
input mapping.
[... ti.map((x)=> x*10, ti.range(10))]
[... ti.map((x, y, z) => [x, y, z], ti.range(5), ti.range(0, 100, 10), ti.range(0, 1000, 100))]
mapcat
Signature: mapcat<T>(fn: (...args: any[]) => Iterable<T>, ...inputs: Iterable<any>[]) => IterableIterator<T>
Like map
, but expects mapping fn to return an iterable result and
produces iterator which yields the flat concatenation of results (only
the first level of nesting is removed). null
or undefined
values
returned by the mapping fn are skipped in the output.
[... ti.mapcat((x) => ti.repeat(x, 3), "hello")]
[...ti.mapcat((x) => x < 5 ? ti.repeat(x,x) : null, ti.range(10))]
mapIndexed
Signature: mapIndexed<T>(fn: (i: number, ...args: any[]) => T[], ...inputs: Iterable<any>[]) => IterableIterator<T>
Like map
, but too passes a monotonically increasing index
as first
argument to mapping fn.
[... ti.mapIndexed((i, a, b) => [i, a, b], "hello", "there")]
maybeIterator
Signature: maybeIterator(x: any) => any
Helper function, returning arg's iterator (if present) or else
undefined
.
maybeObjectIterator
Signature: maybeObjectIterator(x: any) => any
Helper function, checks if x
is object-like (but no generator) and
returns objectIterator(x)
or else undefined
.
objectIterator
Signature: objectIterator(x: any) => IterableIterator<any[]>
Produces iterator of an object"s key/value pairs.
[... ti.objectIterator({a: 23, b: 42, c: [1, 2, 3]})]
partition
Signature: partition<T>(n: number, step: number, input: Iterable<T>, all = false) => IterableIterator<T[]>
Produces iterator of fixed size partitions/chunks of input values.
Produces overlapping partitions, if step
< partition size n
. Unless
the optional all
flag is enabled, returns only completely filled
partitions. If all = true
, the last partition produced may have less
than n
items (though never empty).
[... ti.partition(3, 3, ti.range(10))]
[... ti.partition(3, 3, ti.range(10), true)]
[... ti.partition(3, 1, ti.range(10))]
[... ti.partition(3, 5, ti.range(10))]
partitionBy
Signature: partitionBy<T>(fn: (x: T) => any, input: Iterable<T>) => IterableIterator<T[]>
Produces iterator of partitions/chunks of input values. Applies fn
to
each item and starts new partition each time fn
returns new result.
[... ti.partitionBy((x) => x / 5 | 0, ti.range(11))]
randomSample
Signature: randomSample<T>(prob: number, input: Iterable<T>) => IterableIterator<T>
Produces iterator which consumes input and yields values with given
probability (0 .. 1 range).
[... ti.randomSample(0.1, ti.range(100))]
range
Signature: range(from?: number, to?: number, step?: number) => IterableIterator<number>
Produces iterator of monotonically increasing or decreasing values with
optional step
value.
- If called without arguments, produces values from 0 .. +∞.
- If called with 1 arg: 0 ... n (exclusive)
- If called with 2 arg:
from
... to
(exclusive)
If from
> to
and no step
is given, a step
of -1
is used.
[... ti.take(5, ti.range())]
[... ti.range(5)]
[... ti.range(100, 105)]
[... ti.range(5,0)]
[... ti.range(0, 50, 10)]
[... ti.range(50, -1, -10)]
reduce
Signature: reduce<A, B>(rfn: (acc: B, x: A) => B | ReducedValue<B>, acc: B, input: Iterable<A>) => B
Consumes and reduces input using reduction function rfn
, then returns
reduced result value. rfn
can abort reduction process by returning a
value wrapped using reduced(x)
. If this is done, then this value is
unwrapped and returned as final result.
If input is empty, returns initial acc
umulator arg.
ti.reduce((acc, x) => acc + x, 0, ti.range(10))
ti.reduce((acc, x) => { return acc += x, acc >= 15 ? ti.reduced(acc) : acc }, 0, ti.range())
reductions
Signature: reductions<A, B>(rfn: (acc: B, x: A) => B | ReducedValue<B>, acc: B, input: Iterable<A>) => IterableIterator<B[]>
Like reduce
, but yields an iterator of all intermediate reduction
results. Always yields at least initial acc
umulator arg, even if input
is empty.
Thus, the result is the equivalent of an exclusive scan
operation
(with the exception of possible early bail out via reduced
).
[... ti.reductions((acc, x) => acc + x, 0, ti.range(10))]
[... ti.reductions((acc, x) => { return acc += x, acc >= 15 ? ti.reduced(acc) : acc }, 0, ti.range())]
reduced
Signature: reduced<T>(x: T) => ReducedValue<T>
For use inside reduction functions only. Wraps result in marker type to
cause early termination of reduction (see example above).
repeat
Signature: repeat<T>(x: T, n?: number) => IterableIterator<T>
Produces an iterator of infinite (by default) repetitions of value x
.
If n
is given, produces only that many values.
[... ti.take(5, ti.repeat(42))]
[... ti.repeat(42, 5)]
repeatedly
Signature: repeatedly<T>(fn: () => T, n?: number) => IterableIterator<T>
Produces an iterator of infinite (by default) results of calling the
no-arg fn
repeatedly. If n
is given, produces only that many values.
[... ti.take(3, ti.repeatedly(() => Math.random()))]
[... ti.repeatedly(() => Math.random(), 3)]
reverse
Signature: reverse<T>(input: Iterable<T>) => IterableIterator<T>
Yields iterator lazily producing reverse result order of input
(must be finite). If input is not already array-like (strings are
for this purpose), the function first consumes & caches input as array.
[...ti.reverse([1, 2, 3])]
[...ti.reverse("hello")]
[...ti.reverse(ti.take(10, ti.iterate(x => x * 2, 1)))]
some
Signature: some<T>(pred: (x: T) => boolean, input: Iterable<T>) => T
Consumes iterator and calls pred
for each item. When pred
returns
true
, process stops and returns this first successful item. When none
of the items pass the predicate, the function returns undefined
.
var nums = ti.iterator([1, 2, 3]);
ti.some((x) => (x % 2) === 0, nums);
nums.next()
nums = ti.iterator([1, 2, 3]);
ti.some((x) => x > 3, nums);
nums.next()
take
Signature: take<T>(n: number, input: Iterable<T>) => IterableIterator<T>
Produces iterator of the first n
values of input (or less than n
, if
input is too short...)
[... ti.take(3, ti.range())]
takeNth
Signature: takeNth<T>(n: number, input: Iterable<T>) => IterableIterator<T>
Produces an iterator only yielding every n
th item from input.
[... ti.takeNth(3, ti.range(10))]
takeWhile
Signature: takeWhile<T>(pred: (x: T) => boolean, input: Iterable<T>) => IterableIterator<T>
Produces iterator which calls pred
for each item of input and
terminates as soon as pred
returns false
.
Important: Due to lack of look-ahead in the ES6 iterator API, the
value failing the given pred
will be lost when working with the
original iterator after takeWhile
.
var input = ti.range(10);
[... ti.takeWhile((x)=> x < 5, input)]
[... input]
takeLast
Signature: takeLast<T>(n: number, input: Iterable<T>) => IterableIterator<T>
Consumes input and produces iterator of the last n
values of input (or
less than n
, if input is too short...)
Important: Never attempt to use with infinite inputs!
[... ti.takeLast(5, ti.range(1000))]
[... ti.takeLast(5, ti.range(3))]
walk
- Signature:
walk(fn: (x: any) => void, input: Iterable<any>, postOrder = false) => void
- Signature:
walk(fn: (x: any) => void, children: (x: any) => any, input: Iterable<any>, postOrder = false) => void
Recursively walks input and applies fn
to each element (for in-place
editing or side effects). Only iterable values and objects (but not
strings) are traversed further. Traversal is pre-order by default, but
can be changed to post-order via last arg.
Note: Object traversal is done via objectIterator()
and so will
include pairs of [k, v]
values.
The optional children
fn can be used to select child values of the
currently visited value. If this function is given then only its return
values will be traversed further (with the same constraint as mentioned
above). If the fn returns null
or undefined
, no children will be
visited.
let doc = {
tag: "svg",
content: [
{
tag: "g",
content: [
{ tag: "rect" },
{ tag: "circle" }
]
},
{ tag: "circle", attr: { fill: "red" } }
]
};
let circleTX = x => {
if (x.tag === "circle") {
x.attr = x.attr || {};
x.attr.x = Math.random()*100;
x.attr.y = Math.random()*100;
x.attr.r = 5;
}
};
ti.walk(circleTX, (x) => x.content, doc);
doc.content[0].content[1]
doc.content[1]
walkIterator
- Signature:
walkIterator(input: Iterable<any>, postOrder = false) => IterableIterator<any>
- Signature:
walkIterator(input: Iterable<any>, children: (x: any) => any, postOrder = false) => IterableIterator<any>
Yields an iterator performing either pre-order (default) or post-order
traversal of
input. Only iterable values and objects (but not strings) are traversed
further.
Note: Object traversal is done via objectIterator()
and so will
include pairs of [k, v]
values.
The optional children
fn can be used to select child values of the
currently visited value. If this function is given then only its return
values will be traversed further (with the same constraint as mentioned
above). If the fn returns null
or undefined
, no children will be
visited.
[...ti.map(JSON.stringify, ti.walkIterator([[[1, [2]], [3, [4]]], [5]], false))]
[...ti.map(JSON.stringify, ti.walkIterator([[[1, [2]], [3, [4]]], [5]], true))]
zip
Signature: zip(keys: Iterable<any>, vals: Iterable<any>, target?: any) => any
Takes an iterator of keys and iterator of values, pairwise combines
items from each input and associates them as key-value mappings in a
given target object (if target
is missing, returns new object). Stops
as soon as either input is exhausted.
ti.zip("abcdef", ti.range())
ti.zip(ti.range(5,10), ti.range(100,200), new Uint8Array(16))
var langs=[
{id: "js", name: "JavaScript"},
{id: "clj", name: "Clojure"},
{id: "ts", name: "TypeScript"}
];
ti.zip(ti.map((x)=> x.id, langs), langs)
Authors
If this project contributes to an academic publication, please cite it as:
@misc{thing-iterators,
title = "@thi.ng/iterators",
author = "Karsten Schmidt",
note = "https://thi.ng/iterators",
year = 2017
}
License
© 2017 - 2024 Karsten Schmidt // Apache License 2.0