js-reporters
Advanced tools
Readme
The Common Reporter Interface (CRI) for JavaScript Unit Testing Frameworks.
| Avoid this: | Do this: |
|---|---|
 |  |
https://github.com/js-reporters/js-reporters/issues/
We on the QUnit team have been discussing the possibility of working with other JS test frameworks, especially those that can be run client-side (e.g. Mocha, Jasmine, Intern, Buster, etc.), to agree upon a "Common Reporter Interface" so that we could hopefully share Reporter plugins between testing frameworks. This would also benefit high-level consumers of the frameworks such as Karma, BrowserStack, SauceLabs, Testling, etc.
This would most likely come in the form of:
.on(...)/.off(...)) OR an object with standard "hook" propertiesMyLib.addReporter(x), MyLib.reporter = x;, etc.Would you be interested in discussing this with us further? Please join in!
Based on the discussion in #1, this is the suggested minimum set of event names to be triggered by a testing framework, to be consumed by reporters or other testing tools.
runStart: Indicates the beginning of a testsuite, triggered just once.suiteStart: Triggered at the start of each group of tests within a testsuite.testStart: Triggered at the start of each test.testEnd: Triggered at the end of each test.suiteEnd: Triggered at the end of each group of tests within a testsuite.runEnd: Indicates the end of a testsuite, triggered just once.The criteria for picking these event names included:
The recommended reporting order for emitting the aformentioned events with their related data should be done in source order. Please note that execution order is different from reporting order, that's why we don't assume any specific execution order, but we recommend whatever is higher up in the source file should be emitted first.
For more background check #62.
Based on the discussion in #12, there are two basic data structures: Suites and Tests. A test represents an atomic test/spec/it(). A suite contains tests and optionally other suites.
The structures have been divided in two parts, a start part and an end part:
For testStart and testEnd, the corresponding TestStart, respectively TestEnd, object is passed to the reporter. The same applies to suiteStart and suiteEnd where the matching SuiteStart/SuiteEnd object is passed to the reporter. For runStart and runEnd a "global" suite object is passed to the reporter, this suite wraps all other top-level user defined suites as its child suites, it will be reffered to as globalSuite.
The data structures are defined as follows:
SuiteStart: Object - A SuiteStart is a collection of test-starts and potentially other suite-starts, emitted before the suite have been executed, which means before executing any of its tests and child suites.
String|undefined - name of the suite, will be undefined only for the globalSuite.Array - array of strings containing the name of the suite and the names of all its suites ancestors.Array - array containing all tests that directly belong to the suite (but not to a child suite).Array - array with all direct subsuites.Object - contains the total number of tests in the suite, including the tests of the child suites.
Number - total number of testsSuiteEnd: Object - A SuiteEnd is a collection of test-ends and potentially other suite-ends, emitted after the suite has been executed, which means that all its tests and child suites have been also executed. In addition to the properties of SuiteStart, SuiteEnd also has the following properties:
String - summarized status of the suite.
failed, if at least one test in the suite or in its child suites has failed.skipped, if all tests in the suite and in its child suites are skipped (and there is at least one skipped test).todo, if all tests in the suite and in its child suites are todo (and there is at least one todo test).passed, if there is at least one passed test in the suite or in its child suites and all other tests are skipped or todo, or if there are no tests in the suite.Object - contains how many tests have passed, failed etc. including the tests of child suites.
Number - number of passed tests.Number - number of failed tests.Number - number of skipped tests.Number - number of todo tests.Number - total number of tests, the sum of the above properties must equal this one.Number - execution time of the whole suite in milliseconds (including child suites).The above suite properties apply also for the globalSuite.
TestStart: Object - A test-start holds basic information on a single test/spec before its execution.
String - name of the test.String - name of the suite the test belongs to.Array - array of strings containing the name of the test and the names of all its suites ancestors.1TestEnd: Object - A test-end holds basic information on a single test/spec after its execution.
String - name of the test.String - name of the suite the test belongs to.Array - array of strings containing the name of the test and the names of all its suites ancestors.String - result of the test. Can be:
passed, if all assertions have passed.failed, if at least one assertion has failed or if the test is todo and all assertions passed.skipped, if the test is disabled and wasn't executed.todo, if the test is todo and at least one assertion failed.*Number - execution time in milliseconds.Array - array containing all errors, i.e failed Assertions. It will contain at least one error for failed statuses and it will be empty for statuses other than failed.Array - array of Assertions containing all assertions passed and failed, for a skipped test there will be an empty array. Frameworks that don't track passed assertions can always provide an empty array for passed tests. In that case, for failed tests this should match the errors property.*For more info about todo tests, please refer to the QUnit documentation for todo tests and the TAP 13 specification on the todo directive.
Based on the discussion in #79, js-reporters establishes also a minimum set of properties for the emitted assertions, failed or passed:
Object - An object which contains information of a single assertion/expectation.
Boolean - true for a passed assertion, failed for a failed assertion.* - the actual value passed to the assertion, should coincide with expected for passed assertions.* - the expected value passed to the assertion, should coincide with actual for passed assertions.String - a description.String|undefined - represents the stack trace for a failed assertion, for a passed one it is undefined.Boolean - whether this assertion was part of a todo testAdditional properties (not defined here) can be added to the Assertion object. If Assertion objects are included in the Test and Suite objects defined above, it is recommended to remove the actual and expected values after TestEnd occurs to avoid leaking memory.
For details please check out the docs and especially the example which presents a testsuite and its reporting data based on the standard presented above.
For implementation examples please check Usage of the adapters and Integrations.
Listen to the events and receive the emitted data:
// Attach one of the exiting adapters.
var runner = JsReporters.autoRegister();
// Listen to the same events for any testing framework.
runner.on('testEnd', function(test) {
console.log('Test %s has errors:', test.fullname.join(' '), test.errors);
});
runner.on('runEnd', function(globalSuite) {
var testCounts = globalSuite.testCounts;
console.log('Testsuite status: %s', globalSuite.status);
console.log('Total %d tests: %d passed, %d failed, %d skipped',
testCounts.total,
testCounts.passed,
testCounts.failed,
testCounts.skipped);
console.log('Total duration: %d', globalSuite.runtime);
});
// Or use one of the built-in reporters.
JsReporters.TapReporter.init(runner);
autoRegister()
Auto registers one of the existing adapters by checking for existing testing frameworks in the global scope and returns the runner to attach event listeners. If no framework is found, it will throw an Error.
JsReporters.autoRegister();
This section is dedicated to explain the limitations of the adapters in respect to the standard.
The only limitation is the emitting order, which is not done in source order:
If you want to know more about each testing framework and about their emitting order, please checkout the frameworks document.
FAQs
Common reporter interface for JavaScript testing frameworks.
We found that js-reporters demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 4 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
CISA launched a new project called Vulnrichment to enrich CVEs with details that help prioritize patching and mitigation efforts, as the NVD backlog of unenriched CVEs awaiting analysis surpasses 10,000.
Security News
Socket is joining forces with CISA and other industry leaders at the RSA Conference to sign the Secure by Design pledge, committing to uphold the highest security standards in our products.
Research
The Socket research team breaks down a sampling of malicious packages that download and execute files, among other suspicious behaviors, targeting the popular Discord platform.