solidity-coverage

What is solidity-coverage?

solidity-coverage is an npm package that provides code coverage metrics for Solidity smart contracts. It helps developers ensure that their tests cover all possible execution paths in their smart contracts, which is crucial for identifying potential vulnerabilities and ensuring the robustness of the code.

What are solidity-coverage's main functionalities?

Code Coverage Report

This feature allows you to generate a code coverage report for your Solidity smart contracts. By integrating solidity-coverage with your Truffle configuration, you can run your tests and get a detailed report on which lines of code were executed.

const coverage = require('solidity-coverage');
module.exports = {
  networks: {
    development: {
      host: '127.0.0.1',
      port: 8545,
      network_id: '*',
    },
  },
  plugins: ['solidity-coverage'],
};

Integration with Truffle

solidity-coverage integrates seamlessly with the Truffle framework, allowing you to run your tests and generate coverage reports without additional setup. This integration simplifies the process of ensuring your smart contracts are thoroughly tested.

const coverage = require('solidity-coverage');
module.exports = {
  networks: {
    development: {
      host: '127.0.0.1',
      port: 8545,
      network_id: '*',
    },
  },
  plugins: ['solidity-coverage'],
};

Command Line Interface

You can use the command line interface to run solidity-coverage directly from your terminal. This command will execute your tests and generate a coverage report, making it easy to incorporate into your CI/CD pipeline.

npx solidity-coverage

Other packages similar to solidity-coverage

truffle

Truffle is a development environment, testing framework, and asset pipeline for Ethereum. While it does not provide code coverage out of the box, it can be used in conjunction with solidity-coverage to achieve similar functionality. Truffle focuses more on the overall development and deployment process of smart contracts.

hardhat

Hardhat is a development environment for Ethereum software. It provides a flexible and extensible way to manage and automate the recurring tasks inherent to developing smart contracts and dApps. Hardhat has its own coverage plugin, hardhat-coverage, which offers similar functionality to solidity-coverage but is designed to work specifically with the Hardhat environment.

embark

Embark is a framework for serverless Decentralized Applications using Ethereum, IPFS, and other platforms. It includes a testing framework and can be extended with plugins to provide code coverage. Embark is more comprehensive in scope, covering the entire dApp development lifecycle.

README

solidity-coverage

Code coverage for Solidity testing

• For more details about what this is, how it works and potential limitations, see the accompanying article.
• solidity-coverage is Solcover

Requirements

• Hardhat >= 2.11.0

Install

$ yarn add solidity-coverage --dev

Require the plugin in hardhat.config.js (Hardhat docs)

require('solidity-coverage')

Or, if you are using TypeScript, add this to your hardhat.config.ts:

import 'solidity-coverage'

Resources:

• 0.8.0 release notes

Run

npx hardhat coverage [command-options]

Trouble shooting

Missing or unexpected coverage? Make sure you're using the latest plugin version and run:

$ npx hardhat clean
$ npx hardhat compile
$ npx hardhat coverage

Typescript compilation errors?

$ npx hardhat compile
$ TS_NODE_TRANSPILE_ONLY=true npx hardhat coverage

Weird test failures or plugin conflicts?

# Setting the `SOLIDITY_COVERAGE` env variable tells the coverage plugin to configure the provider
# early in the hardhat task cycle, minimizing conflicts with other plugins or `extendEnvironment` hooks

$ SOLIDITY_COVERAGE=true npx hardhat coverage

Additional Help

• FAQ
• Advanced Topics

Command Options

Option	Example	Description
testfiles	--testfiles "test/registry/*.ts"	Test file(s) to run. (Globs must be enclosed by quotes and use globby matching patterns)
sources	--sources myFolder or --sources myFile.sol	Path to single folder or file to target for coverage. Path is relative to Hardhat's paths.sources (usually contracts/)
solcoverjs	--solcoverjs ./../.solcover.js	Relative path from working directory to config. Useful for monorepo packages that share settings. (Path must be "./" prefixed)
matrix	--matrix	Generate a JSON object that maps which mocha tests hit which lines of code. (Useful as an input for some fuzzing, mutation testing and fault-localization algorithms.) More...

^* Advanced use

Config Options

Additional options can be specified in a .solcover.js config file located in the root directory of your project.

Example:

module.exports = {
  skipFiles: ['Routers/EtherRouter.sol']
};

Option	Type	Default	Description
skipFiles	Array	[]	Array of contracts or folders (with paths expressed relative to the contracts directory) that should be skipped when doing instrumentation.(ex: [ "Routers", "Networks/Polygon.sol"]) :warning: RUN THE HARDHAT CLEAN COMMAND AFTER UPDATING THIS
irMinimum	Boolean	false	Speeds up test execution times when solc is run in viaIR mode. If your project successfully compiles while generating coverage with this option turned on (it may not!) it's worth using
modifierWhitelist	String[]	[]	List of modifier names (ex: onlyOwner) to exclude from branch measurement. (Useful for modifiers which prepare something instead of acting as a gate.))
mocha	Object	{ }	Mocha options to merge into existing mocha config. grep and invert are useful for skipping certain tests under coverage using tags in the test descriptions. More...
measureStatementCoverage	boolean	true	Computes statement (in addition to line) coverage. More...
measureFunctionCoverage	boolean	true	Computes function coverage. More...
measureModifierCoverage	boolean	true	Computes each modifier invocation as a code branch. More...
:printer: OUTPUT
matrixOutputPath	String	./testMatrix.json	Relative path to write test matrix JSON object to. More...
mochaJsonOutputPath	String	./mochaOutput.json	Relative path to write mocha JSON reporter object to. More...
abiOutputPath	String	./humanReadableAbis.json	Relative path to write diff-able ABI data to
istanbulFolder	String	./coverage	Folder location for Istanbul coverage reports.
istanbulReporter	Array	['html', 'lcov', 'text', 'json']	Istanbul coverage reporters
silent	Boolean	false	Suppress logging output
:recycle: WORKFLOW HOOKS
onServerReady*	Function		Hook run after server is launched, before the tests execute. Useful if you need to use the Oraclize bridge or have setup scripts which rely on the server's availability. More...
onPreCompile*	Function		Hook run after instrumentation is performed, before the compiler is run. Can be used with the other hooks to be able to generate coverage reports on non-standard / customized directory structures, as well as contracts with absolute import paths. More...
onCompileComplete*	Function		Hook run after compilation completes, before tests are run. Useful if you have secondary compilation steps or need to modify built artifacts. More...
onTestsComplete*	Function		Hook run after the tests complete, before Istanbul reports are generated. More...
onIstanbulComplete*	Function		Hook run after the Istanbul reports are generated, before the coverage task completes. Useful if you need to clean resources up. More...
:warning: LOW LEVEL
configureYulOptimizer	Boolean	false	Setting to true lets you specify optimizer details (see next option). If no details are defined it defaults to turning on the yul optimizer and enabling stack allocation
solcOptimizerDetails	Object	undefined	Must be used in combination with configureYulOptimizer. Allows you to configure solc's optimizer details. (See FAQ: Running out of stack ).

^* Advanced use

Viewing the reports:

• You can find the Istanbul reports written to the ./coverage/ folder generated in your root directory.

API

Solidity-coverage's core methods and many utilities are available as an API.

const CoverageAPI = require('solidity-coverage/api');

Documentation available here.

Detecting solidity-coverage from another task

If you're writing another plugin or task, it can be helpful to detect whether coverage is running or not. The coverage plugin sets a boolean variable on the globally injected hardhat environment object for this purpose.

hre.__SOLIDITY_COVERAGE_RUNNING === true

Example reports

• openzeppelin-contracts (Codecov)

Funding

You can help fund solidity-coverage development through DRIPS. It's a public goods protocol which helps distribute money to packages in your dependency tree. (It's great, check it out.)

Contribution Guidelines

Contributions are welcome! If you're opening a PR that adds features or options please consider writing full unit tests for them. (We've built simple fixtures for almost everything and are happy to add some for your case if necessary).

Set up the development environment with:

$ git clone https://github.com/sc-forks/solidity-coverage.git
$ yarn