baseline - npm Package Compare versions

Comparing version 0.1.3 to 0.1.4

COPYRIGHT.txt

 /*! *****************************************************************************
  Copyright (c) Artifact Health, LLC. All rights reserved.
+
  Licensed under the Apache License, Version 2.0 (the "License"); you may not use
- this file except in compliance with the License. You may obtain a copy of the
- License at http://www.apache.org/licenses/LICENSE-2.0
+ the files contained within this project except in compliance with the License.
+ You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
 
- THIS CODE IS PROVIDED ON AN *AS IS* BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, EITHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION ANY IMPLIED
+ THIS CODE IS PROVIDED ON AN *AS IS* BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
+ ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION ANY IMPLIED
  WARRANTIES OR CONDITIONS OF TITLE, FITNESS FOR A PARTICULAR PURPOSE,
@@ -14,2 +15,8 @@ MERCHANTABLITY OR NON-INFRINGEMENT.
  and limitations under the License.
+
+ This software is based on or incorporates material and code from the projects
+ Benchmark.js and Mocha (collectively "Third Party Code"). Artifact Health, LLC
+ is not the original author of the Third Party Code. The original copyright
+ notice and the license, under which Artifact Health, LLC received such Third
+ Party Code, are set forth below in NOTICES.txt.
  ***************************************************************************** */

package.json

 {
   "name": "baseline",
   "description": "A benchmarking library that allows performance to be compared to an established baseline.",
-  "version": "0.1.3",
+  "version": "0.1.4",
   "author": {
@@ -6,0 +6,0 @@ "name": "Artifact Health, LLC",

README.md

@@ -8,2 +8,3 @@ # Baseline
 
+* [`A note on micro-benchmarks`](#micro-benchmarks)
 * [`Installation`](#installation)
@@ -16,4 +17,18 @@ * [`Test suites`](#test-suites)
 * [`Hooks`](#hooks)
+* [`Pending tests`](#pending-tests)
+* [`Reporters`](#reporters)
 
 
+<a name="micro-benchmarks" />
+## A note on micro-benchmarks
+
+Baseline cannot be used for micro-benchmarks such as determining if == or === is faster. The time required to call the
+test function is generally greater than the time to execute the test itself. The [Benchmark.js](http://benchmarkjs.com/)
+library is able run micro-benchmarks because it dynamically creates a function, when possible, that compiles the test case into
+the test loop. Baseline does not do this. However, even this technique of dynamically creating the test function is not
+possible for asynchronous tests or tests that reference variables from an outer scope. Regardless, V8 compiler
+optimizations can make micro-benchmarks unreliable. Please see [this video](https://www.youtube.com/watch?v=65-RbBwZQdU)
+for more information.
+
+
 <a name="installation" />
@@ -35,14 +50,19 @@ ## Installation
 ```
-suite("Regexp vs indexOf", function() {
+suite("Sorting Algorithms", function() {
 
-    var str = "hello world!";
-    var reg = /world/;
+    test("Bubble Sort", function() {
+        bubbleSort([49, 344, 431, 144, 122, 8, 207, 49, 8, 481, 10, 2]);
+    });
 
-    test("Regexp", function() {
-        reg.test(str);
+    test("Insertion Sort", function() {
+        insertionSort([49, 344, 431, 144, 122, 8, 207, 49, 8, 481, 10, 2]);
     });
 
-    test("indexOf", function() {
-        str.indexOf("world");
-    });
+    function insertionSort(array) {
+        // implementation of insertion sort
+    }
+
+    function bubbleSort(array) {
+        // implementation of bubble sort
+    }
 });
@@ -56,5 +76,5 @@ ```
 
-Regexp vs indexOf
-  Regexp: 15,297,117 ops/sec ±1.27%
-  indexOf: 21,860,061 ops/sec ±0.68%
+Sorting Algorithms
+  Bubble Sort: 6,668,504 ops/sec ±0.66%
+  Insertion Sort: 1,362,380 ops/sec ±0.74%
 
@@ -84,15 +104,18 @@ Completed 2 tests.
 Once a baseline file has been created, if the program is run with the `-b` option again, the tests will be compared against
-the established baseline. If the change in the performance of a test compared to its baseline is at least 10%,
+the established baseline. If a change in the performance of a test compared to its baseline is at least 10%,
 the test is reported as changed.
 
+Suppose we changed the implementation of the insertion sort algorithm, improving its performance. Running Baseline
+again will result in the following output.
+
 ```
 $ baseline -b results.json test.js
 
-Tests will be compared to baseline established on 2/24/2015 at 10:11:58 PM.
+Tests will be compared to baseline established on 2/26/2015 at 5:25:52 PM.
 
-Regexp vs indexOf
-  Regexp: 13,614,434 ops/sec ±0.73% (11% slower than baseline)
-  indexOf: 21,564,642 ops/sec ±1.11%
+Sorting Algorithms
+  Bubble Sort: 6,733,428 ops/sec ±0.44%
+  Insertion Sort: 1,407,368 ops/sec ±0.70% (19% faster than baseline)
 
-Completed 2 tests, 1 slower.
+Completed 2 tests, 1 faster.
 ```
@@ -125,14 +148,19 @@
 ```
-compare("Regexp vs indexOf", function() {
+compare("Sorting Algorithms", function() {
 
-    var str = "hello world!";
-    var reg = /world/;
+    test("Bubble Sort", function() {
+        bubbleSort([49, 344, 431, 144, 122, 8, 207, 49, 8, 481, 10, 2]);
+    });
 
-    test("Regexp", function() {
-        reg.test(str);
+    test("Insertion Sort", function() {
+        insertionSort([49, 344, 431, 144, 122, 8, 207, 49, 8, 481, 10, 2]);
     });
 
-    test("indexOf", function() {
-        str.indexOf("world");
-    });
+    function insertionSort(array) {
+        // implementation of insertion sort
+    }
+
+    function bubbleSort(array) {
+        // implementation of bubble sort
+    }
 });
@@ -146,5 +174,5 @@ ```
 
-Regexp vs indexOf
-  Regexp: 14,951,082 ops/sec ±1.67% (28% slower)
-  indexOf: 20,680,257 ops/sec ±0.87% (fastest)
+Sorting Algorithms
+  Bubble Sort: 6,709,102 ops/sec ±0.51% (fastest)
+  Insertion Sort: 1,432,421 ops/sec ±0.51% (79% slower)
 
@@ -225,7 +253,7 @@ Completed 2 tests.
 ```
-suite("Regexp vs indexOf", function() {
+suite("Sorting Algorithms", function() {
 
     // other cases
 
-    test.skip("indexOf", function() {
+    test.skip("Insertion Sort", function() {
         // this test will not be executed
@@ -237,2 +265,3 @@ });
 
+<a name="reporters" />
 ## Reporters
@@ -242,3 +271,12 @@
 
-The default reporter outputs results for each test case, including comparison tests.
+The default reporter outputs results for each test.
 
+![Default Reporter](https://raw.githubusercontent.com/artifacthealth/baseline/master/docs/img/default-reporter.png)
+
+
+### Minimal
+
+The minimal reporter only reports results for tests that have changed from baseline. Otherwise, only the
+summary is reported.
+
+![Default Reporter](https://raw.githubusercontent.com/artifacthealth/baseline/master/docs/img/minimal-reporter.png)

No alert changes

Improved metrics

Total package byte prevSize

72874

increased by3.03%

Number of lines in readme file

272

increased by15.74%

Worsened metrics

Number of package files

21

decreased by-4.55%

Lines of code

1342

decreased by-0.74%

No dependency changes