Buffalo-Bench
A benchmarking library that supports async hooks and benchmarks by default.
The problem
This library comes from the problem of handling async hooks in a way that is compatible with benchmarking.
The problem is that async hooks are not supported by Benchmark.js. See the foot notes.
For example, the following code will not work as expected:
new Benchmark('test', async () => {
await doSomething();
}, {
async: true,
async onStart() {
console.log(1);
await new Promise(resolve => setTimeout(resolve, 1000));
console.log(2);
}
})
The previous code will log 1
and then run the benchmark, and the log 2
could be logged before the benchmark is finished or couldn't be logged at all.
This problem prevent us to create an async onStart and/or onComplete for a benchmark like an api call that requires opening a db connection, creating a collection, adding a document and launching a server to handle the request. And after the benchmark finishes clear the databese, close the connection and stop the server.
So, BuffaloBench solves this problem by providing a way to create a benchmarks with all the hooks handled as async by default. (Also works with sync functions)
Examples
Simple example
const Benchmark = require('buffalo-bench');
const bench = new Benchmark("name", async () => {});
const bench = new Benchmark("name", async () => {}, options);
const bench = new Benchmark("name", {fn: async () => {}, ...options});
await bench.run();
Full example:
const bench = new Benchmark('myBenchmark', {
maxTime: 5,
minSamples: 1,
beforeEach: async () => {
await doSomething();
},
afterEach: async () => {
await doSomething();
},
before: async () => {
await doSomething();
},
after: async () => {
await doSomething();
},
onError: async (error) => {
await doSomething();
},
fn: async () => {
await doSomething();
}
});
await bench.run();
Installation
You can get this library as a Node.js module available through the npm registry:
// With npm
$ npm install buffalo-bench
// With yarn
$ yarn add buffalo-bench
Or you can use it standalone in the browser with:
<script src="https://cdn.jsdelivr.net/npm/buffalo-bench"></script>
Api
The Benchmark
constructor takes an options
object argument with the following properties:
maxTime
: The maximum time in seconds that a benchmark can take including hooks.minSamples
: The minimum number of samples that must be taken.beforeEach
: A function to be run once before each benchmark loop, does not count for run time.afterEach
: A function to be run once after each benchmark loop, does not count for run time.before
: A function to be run once before the benchmark loop starts, does not count for run time.after
: A function to be run once after the benchmark loop finishes, does not count for run time.onError
: A function to be run if an error occurs.fn
: The function to be run.
The Benchmark
instance has the following properties:
name
: The name of the benchmark.error
: The error object if an error occurred.cycles
: The number of cycles performed.hz
: The number of cycles per second.meanTime
: The meanTime time per cycle.medianTime
: The medianTime time per cycle.standardDeviation
: The standard deviation.maxTime
: The maximum time.minTime
: The minimum time.times
: An array of times for each cycle.options
: The options object passed to the constructor.stamp
: A timestamp representing when the benchmark was created.runTime
: The total time taken to run the benchmark, this does not include beforeEach, afterEach, before and after hooks.totalTime
: The total time taken to run the benchmark including beforeEach, afterEach, before and after hooks.
The Benchmark
instance has the following methods:
run
: Async method that runs the benchmark.toJSON
: Return a JSON representation of the benchmark.compare
: Compare this benchmark to another.
The Benchmark
class has the following static properties:
version
: A string containing the library version.defaults
: An object containing the default options.
Api Notes
If the beforeEach
afterEach
before
after
onError
returns a Promise, the benchmark will wait for the promise to resolve before continuing.
If the beforeEach
function throws an error, the benchmark will stop and emit an BeforeEachError
event.
If the afterEach
function throws an error, the benchmark will stop and emit an AfterEachError
event.
If the fn
function throws an error, the benchmark will stop and emit an RunError
event.
If the after
function throws an error, the benchmark will stop and emit an AfterError
event.
If the before
function throws an error, the benchmark will stop and emit an BeforeError
event.
If the onError
function throws an error, the benchmark will stop and emit an FatalError
event.
This errors will be found in the error
property of the benchmark instance.
When converting to JSON, the errorMessage
property will be a string containing the error message.
Using typescript
If you want to write your benchmarks with typescript, you can use the library as it is by requiring in your project the buffalo-bench/register
file.
Example:
require('buffalo-bench/register');
require('./my-benchmark.ts');)
import Benchmark from 'buffalo-bench';
const bench = new Benchmark('myBenchmark', () => {});
(async () => {
await bench.run();
console.log(bench.toJSON());
})();
This register file uses the eslint
and pirates
modules to transpile the typescript code to javascript on the fly.
Take into account that this will not check the typescript code for errors. If you want to check your typescript code, you can must use the tsc
package.
Development and Build
- Use
yarn dev
to watch and compile the library on every change to it running the benchmarks in the bench folder. - Use
yarn build
to build the library. - Use
yarn commit
to commit your changes.
Contributing
- Use prettify and eslint to lint your code.
- Add tests for any new or changed functionality.
- Update the readme with an example if you add or change any functionality.
Legal
Author: Masquerade Circus. License Apache-2.0