Testing
Declarative test framework
Install: @travetto/test
npm install @travetto/test
yarn add @travetto/test
This module provides unit testing functionality that integrates with the framework. It is a declarative framework, using decorators to define tests and suites. The test produces results in the following formats:
- TAP 13, default and human-readable
- JSON, best for integrating with at a code level
- xUnit, standard format for CI/CD systems e.g. Jenkins, Bamboo, etc.
Note: All tests should be under the test/.*
folders. The pattern for tests is defined as a regex and not standard globbing.
Definition
A test suite is a collection of individual tests. All test suites are classes with the @Suite decorator. Tests are defined as methods on the suite class, using the @Test decorator. All tests intrinsically support async
/await
.
A simple example would be:
Code: Example Test Suite
import assert from 'assert';
import { Suite, Test } from '@travetto/test';
@Suite()
class SimpleTest {
#complexService: {
doLongOp(): Promise<number>;
getText(): string;
};
@Test()
async test1() {
const val = await this.#complexService.doLongOp();
assert(val === 5);
}
@Test()
test2() {
const text = this.#complexService.getText();
assert(/abc/.test(text));
}
}
Assertions
A common aspect of the tests themselves are the assertions that are made. Node provides a built-in assert library. The framework uses AST transformations to modify the assertions to provide integration with the test module, and to provide a much higher level of detail in the failed assertions. For example:
Code: Example assertion for deep comparison
import assert from 'assert';
import { Suite, Test } from '@travetto/test';
@Suite()
class SimpleTest {
@Test()
async test() {
assert.deepStrictEqual({ size: 20, address: { state: 'VA' } }, {});
}
}
would translate to:
Code: Transpiled test Code
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
const tslib_1 = require("tslib");
const Ⲑ_check_1 = tslib_1.__importStar(require("@travetto/test/src/assert/check.js"));
const Ⲑ_root_index_1 = tslib_1.__importStar(require("@travetto/manifest/src/root-index.js"));
const Ⲑ_decorator_1 = tslib_1.__importStar(require("@travetto/registry/src/decorator.js"));
var ᚕf = "@travetto/test/doc/assert-example.js";
const assert_1 = tslib_1.__importDefault(require("assert"));
const test_1 = require("@travetto/test");
let SimpleTest = class SimpleTest {
static Ⲑinit = Ⲑ_root_index_1.RootIndex.registerFunction(SimpleTest, ᚕf, 1887908328, { test: { hash: 102834457 } }, false, false);
async test() {
Ⲑ_check_1.AssertCheck.check({ file: ᚕf, line: 10, text: "{ size: 20, address: { state: 'VA' } }", operator: "deepStrictEqual" }, true, { size: 20, address: { state: 'VA' } }, {});
}
};
tslib_1.__decorate([
(0, test_1.Test)({ ident: "@Test()", lines: { start: 8, end: 11, codeStart: 10 } })
], SimpleTest.prototype, "test", null);
SimpleTest = tslib_1.__decorate([
Ⲑ_decorator_1.Register(),
(0, test_1.Suite)({ ident: "@Suite()", lines: { start: 5, end: 12 } })
], SimpleTest);
This would ultimately produce the error like:
Code: Sample Validation Error
AssertionError(
message="{size: 20, address: {state: 'VA' }} should deeply strictly equal {}"
)
The equivalences for all of the assert operations are:
assert(a == b)
as assert.equal(a, b)
assert(a !== b)
as assert.notEqual(a, b)
assert(a === b)
as assert.strictEqual(a, b)
assert(a !== b)
as assert.notStrictEqual(a, b)
assert(a >= b)
as assert.greaterThanEqual(a, b)
assert(a > b)
as assert.greaterThan(a, b)
assert(a <= b)
as assert.lessThanEqual(a, b)
assert(a < b)
as assert.lessThan(a, b)
assert(a instanceof b)
as assert.instanceOf(a, b)
assert(a.includes(b))
as assert.ok(a.includes(b))
assert(/a/.test(b))
as assert.ok(/a/.test(b))
In addition to the standard operations, there is support for throwing/rejecting errors (or the inverse). This is useful for testing error states or ensuring errors do not occur.
Throws
throws
/doesNotThrow
is for catching synchronous rejections
Code: Throws vs Does Not Throw
import assert from 'assert';
import { Suite, Test } from '@travetto/test';
@Suite()
class SimpleTest {
@Test()
async testThrows() {
assert.throws(() => {
throw new Error();
});
assert.doesNotThrow(() => {
let a = 5;
});
}
}
Rejects
rejects
/doesNotReject
is for catching asynchronous rejections
Code: Rejects vs Does Not Reject
import assert from 'assert';
import { Suite, Test } from '@travetto/test';
@Suite()
class SimpleTest {
@Test()
async testRejects() {
await assert.rejects(async () => {
throw new Error();
});
await assert.doesNotReject(async () => {
let a = 5;
});
}
}
Error Matching
Additionally, the throws
/rejects
assertions take in a secondary parameter to allow for specification of the type of error expected. This can be:
- A regular expression or string to match against the error's message
- A class to ensure the returned error is an instance of the class passed in
- A function to allow for whatever custom verification of the error is needed
Code: Example of different Error matching paradigms
import assert from 'assert';
import { Suite, Test } from '@travetto/test';
@Suite()
class SimpleTest {
@Test()
async errorTypes() {
assert.throws(() => {
throw new Error('Big Error');
}, 'Big Error');
assert.throws(() => {
throw new Error('Big Error');
}, /B.*Error/);
await assert.rejects(() => {
throw new Error('Big Error');
}, Error);
await assert.rejects(() => {
throw new Error('Big Error');
}, (err: Error) =>
err.message.startsWith('Big') && err.message.length > 4 ? undefined : err
);
}
}
Running Tests
To run the tests you can either call the Command Line Interface by invoking
Terminal: Test Help Output
$ trv test --help
Usage: test [options] [first:string] [regexes...:string]
Options:
-f, --format <string> Output format for test results (default: "tap")
-c, --concurrency <number> Number of tests to run concurrently (default: 7)
-m, --mode <single|standard> Test run mode (default: "standard")
-h, --help display help for command
The regexes are the patterns of tests you want to run, and all tests must be found under the test/
folder.
Travetto Plugin
The VSCode plugin also supports test running, which will provide even more functionality for real-time testing and debugging.
Additional Considerations
During the test execution, a few things additionally happen that should be helpful. The primary addition, is that all console output is captured, and will be exposed in the test output. This allows for investigation at a later point in time by analyzing the output.
Like output, all promises are also intercepted. This allows the code to ensure that all promises have been resolved before completing the test. Any uncompleted promises will automatically trigger an error state and fail the test.