solidity-coverage

solidity-coverage

Code coverage for Solidity testing

• For more details about what this is, how it works and potential limitations, see the accompanying article.
• solidity-coverage is Solcover

Install

$ npm install --save-dev solidity-coverage

Resources:

• 0.7.0 release notes
• A guide to upgrading from 0.6.x to 0.7.x
• 0.6.3 docs

Truffle V5

Add this package to your plugins array in truffle-config.js (Truffle docs)

module.exports = {
  networks: {...},
  plugins: ["solidity-coverage"]
}

Run

truffle run coverage [command-options]

Hardhat

Beginning with v0.7.12, this tool supports Hardhat and runs directly on HardhatEVM.

Require the plugin in hardhat.config.js (Hardhat docs)

require('solidity-coverage')

Run

npx hardhat coverage [command-options]

(Additional Hardhat-specific info can be found here)

Buidler [Deprecated]

Add the plugin in buidler.config.js (Buidler docs)

usePlugin('solidity-coverage')

module.exports = {
  networks: {
    coverage: {
      url: 'http://localhost:8555'
    }
  },
}

Run

npx buidler coverage --network coverage [command-options]

Buidler Project Examples:

• Simple: buidler-metacoin
• More complex: MolochDao/moloch

@openzeppelin/test-environment

OpenZeppelin have written their own coverage generation scripts for test-environment using the solidity-coverage API. A working example can be found at openzeppelin-contracts, here.

Usage notes:

• Coverage runs tests a little more slowly.
• Coverage launches its own in-process ganache server.
• You can set ganache options using the providerOptions key in your .solcover.js config.
• Coverage distorts gas consumption. Tests that check exact gas consumption should be skipped.
• :warning: Contracts are compiled without optimization. Please report unexpected compilation faults to issue 417

Command Options

Option	Example	Description
file	--file="test/registry/*.js"	(Truffle) Filename or glob describing a subset of tests to run. (Globs must be enclosed by quotes.)
testfiles	--testfiles "test/registry/*.ts"	(Buidler) Test file(s) to run. (Globs must be enclosed by quotes.)
solcoverjs	--solcoverjs ./../.solcover.js	Relative path from working directory to config. Useful for monorepo packages that share settings. (Path must be "./" prefixed)
network	--network development	Use network settings defined in the Truffle or Buidler config
temp*	--temp build	:warning: Caution :warning: Path to a disposable folder to store compilation artifacts in. Useful when your test setup scripts include hard-coded paths to a build directory. More...
matrix	--matrix	Generate a JSON object that maps which mocha tests hit which lines of code. (Useful
as an input for some fuzzing, mutation testing and fault-localization algorithms.) More...

^* Advanced use

Config Options

Additional options can be specified in a .solcover.js config file located in the root directory of your project.

Example:

module.exports = {
  skipFiles: ['Routers/EtherRouter.sol']
};

Option	Type	Default	Description
silent	Boolean	false	Suppress logging output
client	Object	require("ganache-core")	Useful if you need a specific ganache version.
providerOptions	Object	{ }	ganache-core options
skipFiles	Array	['Migrations.sol']	Array of contracts or folders (with paths expressed relative to the contracts directory) that should be skipped when doing instrumentation.
measureStatementCoverage	boolean	true	Computes statement (in addition to line) coverage. More...
measureFunctionCoverage	boolean	true	Computes function coverage. More...
measureModifierCoverage	boolean	true	Computes each modifier invocation as a code branch. More...
modifierWhitelist	String[]	[]	List of modifier names (ex: "onlyOwner") to exclude from branch measurement. (Useful for modifiers which prepare something instead of acting as a gate.))
matrixOutputPath	String	./testMatrix.json	Relative path to write test matrix JSON object to. More...
mochaJsonOutputPath	String	./mochaOutput.json	Relative path to write mocha JSON reporter object to. More...
abiOutputPath	String	./humanReadableAbis.json	Relative path to write diff-able ABI data to. More...
istanbulFolder	String	./coverage	Folder location for Istanbul coverage reports.
istanbulReporter	Array	['html', 'lcov', 'text', 'json']	Istanbul coverage reporters
mocha	Object	{ }	Mocha options to merge into existing mocha config. grep and invert are useful for skipping certain tests under coverage using tags in the test descriptions.
onServerReady*	Function		Hook run after server is launched, before the tests execute. Useful if you need to use the Oraclize bridge or have setup scripts which rely on the server's availability. More...
onCompileComplete*	Function		Hook run after compilation completes, before tests are run. Useful if you have secondary compilation steps or need to modify built artifacts. More...
onTestsComplete*	Function		Hook run after the tests complete, before Istanbul reports are generated. More...
onIstanbulComplete*	Function		Hook run after the Istanbul reports are generated, before the ganache server is shut down. Useful if you need to clean resources up. More...

^* Advanced use

API

Solidity-coverage's core methods and many utilities are available as an API.

const CoverageAPI = require('solidity-coverage/api');

Documentation available here.

FAQ

Common problems & questions:

• Running in CI
• Running out of gas
• Running out of time
• Running out of memory
• Why are require statements highlighted as branch points?

Example reports

• metacoin (Istanbul HTML)
• openzeppelin-solidity(Coveralls)

Contribution Guidelines

Contributions are welcome! If you're opening a PR that adds features or options please consider writing full unit tests for them. (We've built simple fixtures for almost everything and are happy to add some for your case if necessary).

Set up the development environment with:

$ git clone https://github.com/sc-forks/solidity-coverage.git
$ yarn

Contributors

• @area
• @cgewecke
• @adriamb
• @cag
• @maurelian
• @rudolfix
• @phiferd
• @e11io
• @elenadimitrova
• @ukstv
• @vdrg
• @andresliva
• @DimitarSD
• @sohkai
• @bingen
• @pinkiebell
• @obernardovieira
• @angus-hamill
• @kandrianov
• @yxliang01
• @maxsam4
• @justinjmoses