testcafe-reporter-allure
This project is an Allure reporter plugin for TestCafé.
The inspiration for this project was its namesake, testcafe-reporter-allure, made by azohra-core, making use of the 2.0.0 version of the allure-js-commons package.
Contents
Installation
This package is namespaced. Therefore the following command can be used to install the reporter in a way that TestCafé can detect it.
(Related ISSUE)
npm install --save-dev testcafe-reporter-allure@npm:@isaac.frontend/testcafe-reporter-allure
The Allure Commandline is needed to convert the Allure-Results into an Allure-Report. This package can be installed with the following command:
npm install --save-dev allure-commandline
Quick install of all dependencies:
npm install --save-dev allure-commandline testcafe testcafe-reporter-allure@npm:@isaac.frontend/testcafe-reporter-allure
Using the reporter
The reporter can then be loaded in two separate ways:
When using TestCafé via the command line, the reporter can be specified by using the --reporter
option.
(Note: The cli option is currently not supported)
testcafe chrome 'path/to/test/file.js' --reporter allure
When using TestCafé via the API, the reporter can be specified by either passing the reporter name or the reporter object in the reporter()
method within the runner.js
file.
testCafe
.createRunner()
.src(['tests/e2e/*.ts'])
.browsers('firefox')
.reporter('allure')
.run();
or
const allureReporter = require('testcafe-reporter-allure');
testCafe
.createRunner()
.src(['tests/e2e/*.ts'])
.browsers('firefox')
.reporter(allureReporter)
.run();
After the Allure-Results are generated, the Allure-Report can be built:
allure generate ./allure/allure-results --clean -o ./allure/allure-report && allure open ./allure/allure-report
Features
Metadata
![Example of the metadata code shown below.](https://github.com/isaaceindhoven/testcafe-reporter-allure/blob/package-distribution/.github/images/metadata.PNG)
Metadata can be added to a test by using the meta()
function. The metadata can be added to both the test
and the fixture
.
Metadata added to a fixture
will be inherited by all tests coupled to that fixture to avoid having to declare metadata that is the same for all tests within the fixture multiple times.
test.meta({
severity: Severity.TRIVIAL,
issue: 'TEST-ISSUE',
description: 'An example discription',
epic: 'Example Epic Ticket',
feature: 'Example Feature Ticket',
story: 'Example Story Ticket',
suite: 'Main Example Group',
otherMeta: 'Example otherMeta parameter.',
})('Example test with metadata', async (t) => {
// Test Code
});
Metadata | Description |
---|
Severity | The severity values are dictated by the allure-js-commons package, these values are: blocker, critical, normal, minor, trivial ; |
Issue | A Jira Issue can be coupled to a test, creating a link within the Allure Report to the Jira Issue page. The URL to the Jira page can be set in the allure-js-commons. |
Epic, Feature, Story | To sort the tests based on the epic , feature , and/or story , these metadata options can be used to form a tree structure. The tree is structured as follows: An epic can have multiple features, a feature can have multiple stories, and a story can have multiple tests. |
Suite | Within the Allure Report, the tests are organized by fixture by default. For a more expansive organization of the tests and fixtures, the suite parameter can be set. When the suite parameter is set within a test , a subcategory is created within the fixture that will group all tests that have the same suite parameter together. When the suite parameter is set within a fixture , a parent category is created that will group multiple fixtures that have the same suite parameter. |
Othermeta | It is also possible to add custom metadata to a test . These will be added as parameters to the Allure Report. These parameters do NOT have to be named otherMeta but do require to be a key-value pair of two strings. For example, adding a color parameter to a test: color: 'black' will result in color: black be added to the final Allure Report. |
Test Steps
![Example of the step code shown below.](https://github.com/isaaceindhoven/testcafe-reporter-allure/blob/package-distribution/.github/images/test-steps.PNG)
With this reporter, test-steps can be defined to split a TestCafé test
into multiple steps. The step function expects three variables: The step name, the TestController, and the actions taken within the step as a TestControllerPromise.
These steps will show up as test-steps within the Allure-Report and will include a screenshot of the page state at the end of the step. These screenshots could then be used to visually follow along with what the test does within each step and get a quick overview where a test might be failing.
import step from 'testcafe-reporter-allure';
test('Example test with steps', async (t) => {
await step('Add developer name to form', t,
t.typeText('#developer-name', 'John Smith')
);
await step('Submit form and check result', t,
t.click('#submit-button')
.expect(Selector('#article-header')
.innerText).eql('Thank you, John Smith!'),
);
});
Configuration
Testcafe-reporter-allure provides a sensible default for the configuration. However, if a different configuration is needed, this default can be overridden by creating the file allure.config.js
and/or allure-categories.config.js
in the root of your project. The allure.config.js
is for the base configuration options, and the allure-categories.config.js
is specifically for editing the categories config used by the Allure Commandline to sort the tests based on pattern matching.
An example allure.config.js
:
module.export = {
RESULT_DIR: './allure/allure-results',
REPORT_DIR: './allure/allure-report',
SCREENSHOT_DIR: './allure/screenshots',
CLEAN_RESULT_DIR: true,
CLEAN_REPORT_DIR: true,
CLEAN_SCREENSHOT_DIR: true,
ENABLE_SCREENSHOTS: true,
ENABLE_QUARANTINE: false,
ENABLE_LOGGING: false,
META: {
SEVERITY: 'Normal',
ISSUE_URL: 'https://jira.example.nl/browse/',
},
LABEL: {
ISSUE: 'JIRA Issue',
FLAKY: 'Flaky test',
SCREENSHOT_MANUAL: 'Screenshot taken manually',
SCREENSHOT_ON_FAIL: 'Screenshot taken on fail',
DEFAULT_STEP_NAME: 'Test step',
},
};
An example allure-categories.config.js
:
module.export = [
{
name: 'Ignored tests',
matchedStatuses: [Status.SKIPPED],
},
{
name: 'Product defects',
matchedStatuses: [Status.FAILED],
messageRegex: '.*Assertion failed.*',
},
{
name: 'Test defects',
matchedStatuses: [Status.FAILED],
},
{
name: 'Warnings',
matchedStatuses: [Status.PASSED],
messageRegex: '.*Warning.*',
},
{
name: 'Flaky tests',
matchedStatuses: [Status.PASSED, Status.FAILED],
messageRegex: '.*Flaky.*',
},
];
License
MIT © ISAAC E-commerce Solutions BV