Hermione is the utility for integration testing of web pages using WebdriverIO and Mocha.
Hermione provides several features which WebdriverIO doesn't and makes testing process easier.
If you are familiar with WebdriverIO and Mocha, you can start writing and running tests in 5 minutes! You need to install hermione via npm and add a tiny config to your project. See details in Quick start section.
When tests are run one by one, it takes a lot of time. Hermione can run tests in parallel sessions in different browsers out of the box.
WebdriverIO provides built-in commands for browser and page manipulation. Often projects need to store some common code and reuse it through all tests. So a developer should create some helpers and include them in the tests.
With hermione it's very simple and straightforward. You can add any number of custom commands in the hermione config and use them as this.browser.myCustomCommand in tests.
Moreover, hermione provides plugins which work as some kind of a hooks. They allow a developer to prepare environment for tests and react properly to test execution events.
Integration tests use a dynamic environment with a lot of dependencies where any of them can work unstable from time to time. As a result, integration tests become red randomly and make them undetermined. It spoils all testing process.
To prevent incidental fails hermione retries a failed test before marking it as a failed. It makes it possible to get rid of a majority of incidental fails. Number of retries can be specified for all browsers or for a specific browser.
:warning: Hermione reruns tests in a new browser session to exclude situations when the browser environment is a cause of the fail.
Sometimes it is needed to run only specific tests but not all tests in a set. Hermione makes it possible. You can specify path to the test file
hermione tests/func/mytest.js
or filter describes by using --grep option
hermione --grep login
or simply use mocha only() API in the test
describe.only('user login', () => {...});
Sometimes you need to skip test not in all browsers but in a specific one. For example, you don't need to run
some test in ugly IE browsers. In hermione you can do it with hermione helper. For example,
you can skip some tests in the specific browser
describe('feature', () => {
hermione.skip.in('ie8', 'it can not work in this browser');
it('nowaday functionality', () => {...});
});
or run tests in one of the browsers
describe('feature', () => {
// will be skipped in all browsers except chrome
hermione.skip.notIn('chrome', 'it should work only in Chrome');
it('specific functionality', () => {...});
});
Hermione has possibility to configure running some set of tests in specific browsers. For example,
specs: [
'tests/common', // run common tests in all browsers specified in the config
{
files: 'tests/desktop',
browsers: ['ie8', 'opera']
},
{
files: 'tests/touch',
browsers: ['iphone', 'android']
}
]
See specs for more details.
All work with a grid client is incapsulated in hermione. Forget about client.init and client.end in your tests ;)
Because of hermione is based on WebdriverIO you need to set up Selenium before proceed further.
The simplest way to get started is to use one of the NPM selenium standalone packages like: vvo/selenium-standalone. After installing it (globally) you can run your server by executing:
selenium-standalone start
This feature allows you to ignore the specified suite or test in any browser with additional comment.
You can do it by using global hermione.skip helper. It supports the following methods:
.in — adds matchers for browsers with additional comment.notIn — .in method with reverted value;Each method takes following arguments:
Note that matchers will be compared with browserId specified in a config file, e.g. chrome-desktop.
For example,
describe('feature', () => {
hermione.skip.in('chrome', "It shouldn't work this way in Chrome");
it('should work this way', () => runTestThisWay());
it('should work that way', () => runTestThatWay());
hermione.skip.in(['chrome', 'firefox', /ie\d*/], 'Unstable test, see ticket TEST-487');
it('should done some tricky things', () => runTrickyTest());
});
in this case behaviour it should work this way will be skipped only in chrome browser, but will be run in other browsers. It should work that way will not be ignored. So skip will be applied only to the nearest test. If you need to skip all tests within a suite you can apply skip helper to a describe - all tests within this suite will be skipped with the same comment.
hermione.skip.in('chrome', 'skip comment');
describe('some feature', () => {
it(...);
it(...);
});
Also you can use .notIn method to invert matching. For example,
// ...
hermione.skip.notIn('chrome', 'some comment');
it('should work this way', () => doSomething());
// ...
in this case test will be skipped in all browsers except chrome.
Methods in and notIn are chainable. So you can skip test in several browsers with different comments. For example,
// ...
hermione.skip
.in('chrome', 'some comment')
.notIn('ie9', 'another comment');
it('test1', () => doSomething());
// ...
If you need to skip test in all browsers without a comment you can use mocha .skip method instead of hermione.skip.in(/.*/);. The result will be the same.
Hermione adds some usefull methods and properties to the webdriverio session after its initialization.
Implemented via two commands:
These methods allow to store some information between webdriver calls so it can be used in custom commands for example. This meta information will be shown in allure report.
Note: hermione saves in meta info last url opened in browser.
Example:
it('test1', function() {
return this.browser
.setMeta('foo', 'bar')
.url('/foo/bar?baz=qux')
.getMeta('foo')
.then((val) => console.log(val)) // prints 'bar'
.getMeta('url')
.then((url) => console.log(url)); // prints '/foo/bar?baz=qux'
});
Execution context can be accessed by browser.executionContext property which contains current test/hook mocha object extended with browser id.
Example:
it('some test', function() {
return this.browser
.url('/foo/bar')
.then(function() {
console.log('test', this.executionContext);
});
});
will print something like this
test: {
"title": "some test",
"async": 0,
"sync": true,
"timedOut": false,
"pending": false,
"type": "test",
"body": "...",
"file": "/foo/bar/baz/qux.js",
"parent": "#<Suite>",
"ctx": "#<Context>",
"browserId": "chrome",
"meta": {},
"timer": {}
}
First of all, make sure that all prerequisites are satisfied.
Install the package.
npm install -g hermione
Then put .hermione.conf.js in the project root.
module.exports = {
specs: ['tests/func'],
browsers: {
chrome: {
desiredCapabilities: {
browserName: 'chrome'
}
}
}
};
Write your first test.
var assert = require('chai').assert;
describe('yandex', function() {
it('should find itself', function() {
return this.browser
.url('https://yandex.com')
.setValue('.search2__input input', 'yandex')
.click('.search2__button button')
.getText('.z-entity-card__title')
.then(function(title) {
assert.equal(title, 'Yandex')
});
});
});
Finally, run tests.
hermione
hermione is tuned using a configuration file. By default .hermione.conf.js is used but a path to the configuration file can be specified using --conf option.
There are only two required fields: specs and browsers.
module.exports = {
specs: [
'tests/desktop',
'tests/touch'
],
browsers: {
chrome: {
desiredCapabilities: {
browserName: 'chrome'
}
}
}
};
Required. The list of paths where hermione will look for tests.
For example,
specs: [
{ // run tests associated with this path in all browsers
files: 'tests/desktop' // which are configured in option `browsers`
},
'tests/deskpad', // the alias for the previous case
{
files: 'tests/desktop/*.hermione.js' // run tests matched with a mask
},
{
files: 'tests/touch', // run tests associated with this path in a browser with id `browser`
browsers: ['browser'] // which is configured in option `browsers`
}
]
Required. The list of browsers which should be used for running tests.
Browser section has the following format
browsers: {
<browser_id>: {
<option>:<value>
<option>:<value>
}
}
<browser-id> values is used for browser identification.
Available browser options:
| Option name | Description |
|---|---|
desiredCapabilities | Required. Used WebDriver DesiredCapabilities |
sessionsPerBrowser | Number of sessions which are run simultaneously. Default value is 1. |
retry | How many times test should be rerun. Default value is 0. |
Selenium grid URL. Default value is http://localhost:4444/wd/hub.
Base service-under-test url. Default value is http://localhost.
Timeout for text execution. Default value is 60000 ms.
Timeout for web page events. Default value is 1000 ms.
If test execution time is greater than this value, then test is slow. Default value is 10000.
Turn webdriver debug mode on. Default value is false.
Number of sessions which are run simultaneously. Global value for all browsers. Default value is 1.
How many times test should be retried in case of a fail. Global value for all browsers. Default value is 0.
Hermione plugins are commonly used to extend built-in possibilities. For example, hermione-allure-reporter and hermione-tunnel.
Plugin is a module which exports a single function. The function has two arguments:
Plugins will be loaded before hermione runs tests.
It's strongly recommended to name hermione plugins with hermione- prefix. It makes search for user plugins very simple.
If a plugin name starts with hermione-, then the prefix can be ommited in the configuration file. If two modules with names hermione-some-module and some-module are specified, then module with prefix will have higher priority.
For example.
// .hermione.conf.js
// ...
plugins: {
'my-cool-plugin': {
param: 'value'
}
}
// ...
// hermione-my-cool-plugin/index.js
module.exports = function(hermione, opts) {
hermione.on(hermione.events.RUNNER_START, function() {
return setUp(hermione.config, opts.param); // config can be mutated
});
hermione.on(hermione.events.RUNNER_END, function() {
return tearDown();
});
}
Properties of hermione object
| Property name | Description |
|---|---|
config | Config which is used in test runner. Can be mutated. |
events | Events list for subscription |
Available events
| Event | Description |
|---|---|
RUNNER_START | Will be triggered before tests execution. If a handler returns a promise, tests will be executed only after promise is resolved. |
RUNNER_END | Will be triggered after tests execution. If a handler returns a promise, tests will be executed only after promise is resolved. |
SESSION_START | Will be triggered after browser session initialization. If a handler returns a promise, tests will be executed only after promise is resolved |
SESSION_END | Will be triggered after browser session quit. If a handler returns a promise, tests will be executed only after promise is resolved |
SUITE_BEGIN | Test suite is about to execute |
SUITE_END | Test suite execution is finished |
SUITE_FAIL | Suite failed. For instance, before hook failed or a browser can not be launched (in fact, browsers are launched in before hook implicitly in the core of hermione) |
TEST_BEGIN | Test is about to execute |
TEST_END | Test execution is finished |
TEST_PASS | Test passed |
TEST_FAIL | Test failed |
TEST_PENDING | Test is skipped |
RETRY | Test failed but went to retry |
ERROR | Generic (no tests) errors. |
INFO | Reserved |
WARNING | Reserved |
EXIT | Will be triggered when SIGTERM is received (for example, Ctrl + C). Handler can return a promise. |
Extra options for mocha which are passed to mocha.setup. See Mocha documentation for the list of options.
mochaOpts: {
ignoreLeaks: true
}
Prepare browser session before tests are run. For example, adding custom user commands.
prepareBrowser: function(browser) {
// do setup here
}
browser argument is a WebdriverIO session.
Configuration data can be changed depending on extra conditions in prepareEnvironment function.
Usage: hermione [options]
Options:
-h, --help Output usage information
-c, --conf <path> Path to configuration file [./.hermione.conf.js]
--baseUrl <url> Base service-under-test url [http://localhost]
--grid <url> Selenium grid URL [http://localhost:4444/wd/hub]
--wait-timeout <ms> Timeout for web page events [10000]
--screenshot-path <path> Path for saving screenshots []
--debug <boolean> Turn webdriver debug mode on [false]
--grep <grep> Filter tests matching string or regexp
-r, --reporter <reporter> Reporter [flat]
-b, --browser <browser> Run test in a specific browser
For example,
hermione --baseUrl http://yandex.ru/search
Note. All CLI options override config values.
Skip browsers specified in a config by passing browser ids. Several browser ids should be split by commas (spaces after commas are allowed).
For example,
HERMIONE_SKIP_BROWSERS=ie10,ie11 hermione
0.12.0 - 2016-08-22
SESSION_START and SESSION_END eventsFAQs
Tests framework based on mocha and wdio
The npm package hermione receives a total of 252 weekly downloads. As such, hermione popularity was classified as not popular.
We found that hermione demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 14 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Newly introduced telemetry in devenv 1.4 sparked a backlash over privacy concerns, leading to the removal of its AI-powered feature after strong community pushback.
Security News
TC39 met in Seattle and advanced 9 JavaScript proposals, including three to Stage 4, introducing new features and enhancements for a future ECMAScript release.
Security News
Deno 2.2 enhances Node.js compatibility, improves dependency management, adds OpenTelemetry support, and expands linting and task automation for developers.