speedy

speedy

Tiny benchmark utility

If you need to benchmark some sort of code or if you are writing a module and want to see how well it performs in comparison with older versions, you can use speedy. Use it for rapid prototyping and informal benchmarks. Don't use it for serious jobs like benchmarking HTTP servers.

This module doesn't check for errors so make sure your code doesn't break before running the benchmark.

The benchmark is formed by tests and each test is formed by samples (default 3). Each sample is executed with a fixed amount of time (default 1000ms). An arithmetic mean and a standard error is calculated with these samples.

A recursive loop is used to measure the speed of the code. There are basically 2 ways to make a recursive loop: nextTick and setImmediate. nextTick cannot loop indefinitely because it has a maximum call stack limit (process.maxTickDepth) and setImmediate is slower than nextTick (see examples/nexttick-vs-setimmediate-vs-setTimeout.js) and produces inconsistent benchmark results. The solution is to use an hybrid approach (found in the node-bench source code):

var async = (function (){
  var i = 0;
  return function (fn){
    if (i++ < 100){
      process.nextTick (fn);
    }else{
      setImmediate (fn);
      i = 0;
    }
  }
})();

async (function (){
  //Code
});

This worked for me and produces consistent benchmark results. But it has a problem. Cannot execute codes that make an excessive use of nextTick in a recursive way because the process.maxTickDepth limit can be reached. In Node.js v0.12 this limit will be removed.

Objects

• Speedy

Speedy

A Speedy instance is returned when the module is required.

Events

• end
• progress
• test

Methods

• Speedy#run([name][, fn][, options]) : Speedy
• Speedy#samples([n]) : undefined | Number
• Speedy#timeout([n]) : undefined | Number

end

Emitted when the benchmark finishes. The callback receives an array. Each index stores an object with he result of each test.

Properties:

• raw - Array
  Stores all the sample results.
• name - String
  Stores the name of the function, if any. It is like the result of the test event.

For example, a baseline benchmark with default options (samples 3, timeout 1000):

speedy
    .on ("end", function (data){
      console.log (data);
      
      /*
      [{
        name: "fn",
        raw: [ 226159317.37112886, 225275805.0081939, 226232582.25479224 ]
      }]
      */
    })
    .run (function fn (){});

progress

Emitted when a sample has been executed. The callback receives a number between (0, 1]. 1 means that the benchmark is finished.

This event is emitted before the test and end events.

test

Emitted for each test. The callback receives an object with the following properties:

• name - String
  The name of the function. If the function is anonymous this property is not available.
• raw - Array
  Array with the sample results. The number of samples can be changed with the samples() function.

This event is emitted before the end event.

Speedy#run([name][, fn][, options]) : Speedy

Executes the benchmark. The results are displayed with an arithmetic mean of operations per second followed by the standard error in percentage. The error should be less than 1%, typically less than 0.5%. If the error is greater than 1% then something went wrong and the mean is not as accurate as it can be.

Options:

• output - WritableStream
  By default the results will be sent to the stdout but any writable stream can be used.

  //The output is sent to a file
  speedy.run (function (){}, { output: require ("fs").createWriteStream ("file") });
  

Asynchronous benchmarking

Simply execute the callback when you are ready to iterate again. The asynchronous benchmarking is a little less precise than the synchronous version.

speedy.run (function (done){
  //...
  done ();
});

Ways to run the benchmark and its results

Anonymous function:

speedy.run (function (){});

/*
<value>
*/

Named function:

speedy.run (function fn (){});

/*
fn
  <value>
*/

Using the name parameter:

speedy.run ("fn", function (){});

/*
fn
  <value>
*/

Batch:

speedy.run ({
  a: function (){},
  b: function (){},
  c: function (){}
});

/*
a
  <value>
b
  <value>
c
  <value>
*/

Speedy#samples([n]) : undefined | Number

Changes or returns the number of samples per test. With more samples the final result will be more stable. An arithmetic mean is calculated with all the samples. Default is 3.

speedy.samples (10);

Speedy#timeout([n]) : undefined | Number

Changes or returns the execution time per sample, in milliseconds. Higher values tend to imply more precise results. Default is 1000.

speedy.timeout (2000);