Synpress is e2e testing framework
based on Cypress.io and
playwright with support for
metamask.
Synpress makes sure to always use latest version of metamask and puts a lot of
effort to make sure that dapp tests are stable and trustful.
It also provides an easy way to use and access metamask straight from your e2e
tests with all features of cypress and playwright.
Usage examples:
For additional custom commands and their examples,
check here.
To see in which direction Synpress is headed to, take a look at
planning board.
Features
- added support for metamask π¦
- supports headless mode thanks to
docker π³
- recommended for local development and
CI
- includes VNC and noVNC
- integrated video recoding π₯ (full screen)
- exposes noVNC with ngrok (optional)
- easy to debug π
- improved error handling
- supports cypress and
playwright debuggers
- noVNC allows for interactions through browser π
- debug remote machines on CI with ngrok
- blazingly-fast β‘
- extensible βοΈ (add own custom commands and plugins)
- can be used in existing
cypress setup
- supports dotenv
- loads all env vars from your
.env
file automatically (from project root
folder)
- ability to use latest metamask or lock it's version to avoid unexpected
failures related to metamask updates
- supports multi-lang of metamask, it doesn't depend on any labels
- synpress is
fully tested
- waits for XHR requests, navigations and animations automatically
- ability to fail test run if there are any browser console errors found
- types support for all additional custom commands
- the best possible options set up in place to avoid flakiness
- etherscan API helpers in place which for ex. allows to compare your
transaction results with etherscan and check tx status
- synthetix helpers in place which allows to interact with synthetix protocol
programatically
- supports codespaces
- run your tests in docker
- get your feedback remotely thanks to ngrok
- use mpeg-4 preview plugin to watch videos from inside codespaces :) ...
π· Example setup for eslint and tsconfig
Project structure:
project_dir
βββ src
βββ tests
βββ e2e
βββ .eslintrc.js
βββ support.js
βββ tsconfig.json
βββ specs
βββ example-spec.js
βββ pages
βββ example-page.js
- Create
.eslintrc.js
inside your tests folder (/project_dir/tests/e2e
):
const path = require('path');
const synpressPath = path.join(
process.cwd(),
'/node_modules/@synthetixio/synpress',
);
module.exports = {
extends: `${synpressPath}/.eslintrc.js`,
};
- Create
support.js
inside your tests folder (/project_dir/tests/e2e
):
import '@synthetixio/synpress/support/index';
^ hint: you can also use this file to extend synpress - add custom commands,
and more..
- Create
tsconfig.json
inside your tests folder (/project_dir/tests/e2e
):
{
"compilerOptions": {
"allowJs": true,
"baseUrl": "../../node_modules",
"types": [
"cypress",
"@synthetixio/synpress/support",
"cypress-wait-until",
"@testing-library/cypress"
],
"outDir": "./output"
},
"include": ["**/*.*"]
}
- You're done! π
To change specific values in default config, you can use --config
flag. For
example, to change path for support.js
file, you can use
synpress run --config "supportFile=__tests__/e2e/supportFile.js"
If you would like to use custom paths for your tests and configs, you should
mirror (full) default synpress config
and then modify it for your needs. Then you can direct synpress to use it with
--configFile
flag.
For example: synpress run --configFile __tests__/e2e/customConfig.config.js
β‘ Important
Synpress doesn't seem to communicate with metamask properly if
"chromeWebSecurity": false
flag is set. More about it
here.
Tests work only in non-headless mode because extensions are not supported in
headless mode in playwright and
Cypress.
As a workaround, use provided docker π³ containers. They solve this issue.
There is a global
before()
which runs metamask setup before all tests:
- passes welcome page
- imports wallet
- changes network (defaults to
kovan
) or creates custom network and changes to
it (depending on your setup) - switches back to Cypress window and starts testing
It requires environmental variable called SECRET_WORDS
to be present in
following format => 'word1, word2, etc..'
or private key in an environmental
variable called PRIVATE_KEY
.
To change default network (kovan
), you can use NETWORK_NAME
environmental
variable, for example: NETWORK_NAME=rinkeby
.
Available choices are: mainnet
, ropsten
, kovan
, rinkeby
, goerli
and
localhost
.
To create and switch to custom network at metamask setup phase, use these:
NETWORK_NAME
=> ex: synthetix
RPC_URL
=> ex: https://synthetix-node.io
CHAIN_ID
=> ex: 123
SYMBOL
(optional) => ex: SNX
BLOCK_EXPLORER
(optional) => ex: https://synthetix-explorer.io
IS_TESTNET
(optional) => ex: false
Metamask version is hardcoded and frequently updated under supervision to avoid
a case when e2e tests break because of CSS classes changes in new version, so
all you need is to keep synpress updated in your project. However, you can still
override metamask with METAMASK_VERSION
environmental variable, for example:
METAMASK_VERSION=9.3.0
or METAMASK_VERSION=latest
.
If you don't want to use environmental variables, you can modify
setupMetamask()
to following:
setupMetamask(secretWordsOrPrivateKey, network, password)
, for example:
setupMetamask('word1, word2, etc..', 'mainnet', 'password')
.
You can also add and switch to custom network by passing an object
instead of
string
inside setupMetamask(secretWordsOrPrivateKey, network, password)
function for network
parameter.
If you want to use Etherscan API helpers, you will have to provide Etherscan API
key using ETHERSCAN_KEY
enironmental variable.
To fail a test if there are any browser console errors, set FAIL_ON_ERROR
to
1
or true
.
Automatic waiting for XHR requests to finish before tests start can be turned on
with CYPRESS_RESOURCES_WAIT
environmental variable, set it to 1
or true
.
If you want to skip metamask extension installation or metamask setup, you can
use SKIP_METAMASK_INSTALL
and SKIP_METAMASK_SETUP
separately. Both variables
accept 1
or true
.
Synpress is blazingly-fast β‘ by default! If you want to change that, you can
use STABLE_MODE=true
(which will introduce delays only between main actions,
300ms by default) / STABLE_MODE=<value>
or SLOW_MODE=true
(which will
introduce delay between every action, 50ms by default) / SLOW_MODE=<value>
.
SYNDEBUG=1
is very useful while debugging your tests. It enables following
features:
π³ Using with Docker
Dreaming about "headless" mode? Here comes a rescue π!
Docker is awesome for CI and local development. Give it a try.
Requirements
Some neat features
- based on docker-e2e β€
- full screen video recording π₯ (together with metamask extension)
- VNC & noVNC support π₯οΈ (very easy to debug with browser)
- ngrok π integration (exposes noVNC for everyone)
- remote: https://.ngrok.io/vnc.html?autoconnect=true (check logs for
url)
How to use
git clone git@github.com:Synthetixio/synpress.git
cd synpress
- (optional) Fill env vars inside
.env
file - (without ngrok)
docker-compose up --build --exit-code-from synpress
- (with ngrok)
docker-compose --profile ngrok up --build --exit-code-from synpress
or
./start-tests.sh
All examples of setup are present in this repository. Just take a look around.
Warning: M2 is not supported with docker. As a workaround - you can use
codespaces, they're fully supported! :)
CI tips & tricks
- use docker-e2e
- stick to bigger resolutions (to make sure that whole extension is visible on
screen when opened)
- synpress is tested on 1920x1080
- take a look at this
example config
π§ͺ Usage
synpress run
to run testssynpress open
to open Cypress UI (may be bugged in some cases because it
doesn't clear metamask state before each e2e test, please use synpress run
)
Command line interface (synpress help
):
Usage: synpress run [options]
launch tests
Options:
-b, --browser <name> run on specified browser (default: "chrome")
-c, --config <config> set configuration values, separate multiple values with a comma
-cf, --configFile <path> specify a path to *.js file where configuration values are set
-e, --env <env=val> set environment variables, separate multiple values with comma
-s, --spec <path or glob> run only provided spec files
-ne, --noExit keep runner open after tests finish
-pr, --project <path> run with specific project path
-q, --quiet only test runner output in console
-r, --reporter <reporter> specify mocha reporter
-ro, --reporterOptions <options> specify mocha reporter options, separate multiple values with comma
-r, --record [dashboard] record video of tests running after setting up your project to record
-k, --key <key> [dashboard] set record key
-p, --parallel [dashboard] run recorded specs in parallel across multiple machines
-g, --group <name> [dashboard] group recorded tests together under a single run
-t, --tag <name> [dashboard] add tags to dashboard for test run
-h, --help display help for command
Usage: synpress open [options]
launch test runner UI
Options:
-cf, --configFile <path> specify a path to *.js file where configuration values are set
-h, --help display help for command
π’ Release process
- Create PR from
dev
branch to master
branch - Merge it (new
-beta
version is automatically released) - Run GitHub Action workflow named
Release CI
with
patch|minor|major
depending on your needs to promote your build.
Alternatively, instead of running GitHub Action for release, you can move on
with manual release process:
- Switch to
master
branch and pull latest changes - Run
yarn release:patch/minor/major
command - Keep
dev
branch up to date with master
Above actions will lead to:
- New npm node module release
- New GitHub packages node module release
- New GitHub release (tagged) created with changelog from commit messages
π More resources