autobench
Automated benchmark avoiding regression in HTTP Applications.
Wrap autocannon
and autocannon-compare
in a box to automatize and monitor HTTP routes.
Installation
This is a Node.js module available through the npm registry. It can be installed using the npm
or yarn
command line tools.
npm i autobench
or globally
npm i -g autobench
Usage
autobench
npx autobench
Add environment DEBUG=autobench:*
to see the log applications. Example:
DEBUG=autobench:debug autobench compare
DEBUG=autobench:info autobench compare
DEBUG=autobench:* autobench compare
Config file
In order to use the autobench
, the project must have a autobench.yml
as config file.
The config file parameters are described bellow:
name: 'Autobench Example'
benchFolder: 'bench'
url: 'http://localhost:3000'
connections: 10
pipelining: 1
duration: 30
benchmarks:
- name: 'request 1'
path: '/'
method: 'POST'
headers:
Content-type: 'application/json'
body:
example: 'true'
email: 'hey-[<id>]@example.com'
idReplacement: true
- name: 'request 2'
path: '/slow'
See autobench.yml
file to examples.
Compare
Command to perform benchmark and compare to the stored benchmark.
It's required to have a previous benchmark stored in the benchFolder
. See Autobench Create to realize it.
Options:
Option | Description | Full command |
---|
-s | When is identified a Performance Regression a autobench-review.md file is created with the summary | autobench compare -s |
autobench compare [-s]
The autobench-review.md
looks like:
## Performance Regression ⚠️
---
The previous benchmark for request-1 was significantly performatic than from this PR.
- **Router**: request-1
- **Requests Diff**: 10%
- **Throughput Diff**: 10%
- **Latency Diff**: 10%
---
The previous benchmark for request-2 was significantly performatic than from this PR.
- **Router**: request-2
- **Requests Diff**: 20%
- **Throughput Diff**: 20%
- **Latency Diff**: 20%
Create
Command to store/override the results in the benchFolder
.
Usually, it should be used to update the to latest benchmarking result, for instance, after each PR merged.
autobench create
Examples
See autobench-example for further details.