[!NOTE]
This is one of 192 standalone projects, maintained as part
of the @thi.ng/umbrella monorepo
and anti-framework.
🚀 Please help me to work full-time on these projects by sponsoring me on
GitHub. Thank you! ❤️
About
Benchmarking & profiling utilities w/ various statistics & formatters (CSV, JSON, Markdown etc.).
Though no public API change (only additions), since v2.0.0 this library
internally (via
now()
) attempts to
use high-res ES
BigInt
timestamps (in Node via
process.hrtime.bigint()
)
or falls back to performance.now()
or lacking that to Date.now()
. In all
cases, returns a (possibly rounded) nanosec-scale timestamp, either as bigint
or number
. The
timeDiff()
function can be used to compute the difference between two such timestamp and
return it as milliseconds.
Status
STABLE - used in production
Search or submit any issues for this package
Related packages
- @thi.ng/csv - Customizable, transducer-based CSV parser/object mapper and transformer
- @thi.ng/markdown-table - Markdown table formatter/generator with support for column alignments
Installation
yarn add @thi.ng/bench
ESM import:
import * as bench from "@thi.ng/bench";
Browser ESM import:
<script type="module" src="https://esm.run/@thi.ng/bench"></script>
JSDelivr documentation
For Node.js REPL:
const bench = await import("@thi.ng/bench");
Package sizes (brotli'd, pre-treeshake): ESM: 2.13 KB
Dependencies
Usage examples
Several projects in this repo's
/examples
directory are using this package:
Screenshot | Description | Live demo | Source |
---|
| Doodle w/ K-nearest neighbor search result visualization | Demo | Source |
| K-nearest neighbor search in an hash grid | Demo | Source |
| Poisson-disk shape-aware sampling, Voronoi & Minimum Spanning Tree visualization | Demo | Source |
| Markdown to Hiccup to HTML parser / transformer | Demo | Source |
| Parser grammar livecoding editor/playground & codegen | Demo | Source |
| Interactive pixel sorting tool using thi.ng/color & thi.ng/pixel | Demo | Source |
| Full umbrella repo doc string search w/ paginated results | Demo | Source |
| Fork-join worker-based raymarch renderer (JS/CPU only) | Demo | Source |
API
Generated API docs
Basic usage
const fib = (n: number) =>
n > 2
? fib(n - 1) + fib(n - 2)
: n > 0
? 1
: 0;
const fib2 = (n: number) => {
const res = [0, 1];
for(let i = 2; i <= n; i++) {
res[i] = res[i - 1] + res[i - 2];
}
return res[n];
};
import { timed, bench } from "@thi.ng/bench";
<<test-functions>>
console.log(timed(() => fib(40)));
console.log(timed(() => fib2(40)));
console.log(bench(() => fib(10), 1e6));
console.log(bench(() => fib2(10), 1e6));
Benchmarking with statistics
The benchmark()
function executes a number of warmup runs, before
executing the main measurement and producing a number of useful
statistics: mean, median, min/max, 1st/3rd quartile, standard deviation
(as percentage)...
See
api.ts
for configuration options.
Also see the formatting section below for other output
options. This example uses the default format...
import { benchmark } from "@thi.ng/bench";
<<test-functions>>
benchmark(() => fib(40), { title: "fib", iter: 10, warmup: 5 });
Benchmark suites
Multiple benchmarks can be run sequentially as suite (also returns an array of
all results):
import { suite, FORMAT_MD } from "@thi.ng/bench";
<<test-functions>>
suite(
[
{ title: "fib2(10)", fn: () => fib2(10) },
{ title: "fib2(20)", fn: () => fib2(20) },
{ title: "fib2(30)", fn: () => fib2(30) },
{ title: "fib2(40)", fn: () => fib2(40) },
],
{ iter: 10, size: 100000, warmup: 5, format: FORMAT_MD }
)
Same table as actual Markdown:
Title | Iter | Size | Total | Frequency | Mean | Median | Min | Max | Q1 | Q3 | SD% |
---|
fib2(10) | 10 | 100000 | 93.25 | 10723774.45 | 9.33 | 9.25 | 8.94 | 10.27 | 9.03 | 9.46 | 4.15 |
fib2(20) | 10 | 100000 | 110.73 | 9030823.33 | 11.07 | 11.02 | 10.91 | 11.56 | 10.92 | 11.10 | 1.76 |
fib2(30) | 10 | 100000 | 175.10 | 5711056.26 | 17.51 | 17.58 | 17.03 | 17.65 | 17.50 | 17.60 | 0.96 |
fib2(40) | 10 | 100000 | 200.01 | 4999765.64 | 20.00 | 19.71 | 19.34 | 21.78 | 19.55 | 19.91 | 3.90 |
Output formatting
The following output formatters are available. Custom formatters can be easily
defined (see source for examples). Formatters are configured via the format
option given to benchmark()
or suite()
.
FORMAT_DEFAULT
- default plain text formattingFORMAT_CSV
- Comma-separated values (w/ column header)FORMAT_MD
- Markdown table format
Profiling
Since v3.3.0 the package also provides a basic profiler to take named
measurements and compute derived statistics. The profiler can by dynamically
enabled/disabled, supports recursion and estimates/subtracts its internal
overhead. Results can be obtained as JSON objects or CSV.
import { Profiler } from "@thi.ng/bench";
const profiler = new Profiler({ warmup: 1e6 });
const countdown = (n, acc = []) => {
profiler.start("countdown");
if (n > 0) countdown(n - 1, (acc.push(n),acc));
profiler.end("countdown");
return acc;
}
countdown(10);
countdown(5);
profiler.deref()
console.log(profiler.asCSV())
Authors
If this project contributes to an academic publication, please cite it as:
@misc{thing-bench,
title = "@thi.ng/bench",
author = "Karsten Schmidt",
note = "https://thi.ng/bench",
year = 2018
}
License
© 2018 - 2024 Karsten Schmidt // Apache License 2.0