Monocart Coverage Reports
đ English | įŽäŊä¸æ
A JavaScript code coverage tool to generate native V8 reports or Istanbul reports.
Usage
It's recommended to use Node.js 20+.
npm install monocart-coverage-reports
const MCR = require('monocart-coverage-reports');
const mcr = MCR({
name: 'My Coverage Report - 2024-02-28',
outputDir: './coverage-reports',
reports: ["v8", "console-details"],
cleanCache: true
});
await mcr.add(coverageData);
await mcr.generate();
Using import
and load options from config file
import { CoverageReport } from 'monocart-coverage-reports';
const mcr = new CoverageReport();
await mcr.loadConfig();
For more information, see Multiprocessing Support
mcr node my-app.js -r v8,console-details
For more information, see Command Line
Options
Available Reports
V8 build-in reports (V8 data only):
v8
- Features:
- A Brand-New V8 Coverage Report User Interface
- Support for Native Byte Statistics
- Support processing big data with high performance
- Coverage for Any Runtime Code
- CSS Coverage Support
- Better Support for Sourcemap Conversion
- Demos: V8 and more
Istanbul build-in reports (both V8 and Istanbul data):
clover
cobertura
html
html-spa
json
json-summary
lcov
lcovonly
none
teamcity
text
text-lcov
text-summary
Other build-in reports (both V8 and Istanbul data):
-
codecov
Save coverage data to a json file with Codecov format (defaults to codecov.json
), see example.
-
codacy
Save coverage data to a json file with Codacy API format (defaults to codacy.json
).
-
console-summary
shows coverage summary in the console.
console-details
Show coverage details in the console. Like text
, but for V8. For Github actions, we can enforce color with env: FORCE_COLOR: true
.
markdown-summary
Save coverage summary to a markdown file (defaults to coverage-summary.md
). For Github actions, we can show the markdown content to a job summary
cat path-to/coverage-summary.md >> $GITHUB_STEP_SUMMARY
-
markdown-details
Save coverage details to a markdown file (defaults to coverage-details.md
).
-
raw
only keep all original data, which can be used for other reports input with inputDir
. see Merge Coverage Reports
-
Custom Reporter
{
reports: [
[path.resolve('./test/custom-istanbul-reporter.js'), {
type: 'istanbul',
file: 'custom-istanbul-coverage.text'
}],
[path.resolve('./test/custom-v8-reporter.js'), {
type: 'v8',
outputFile: 'custom-v8-coverage.json'
}],
[path.resolve('./test/custom-v8-reporter.mjs'), {
type: 'both'
}]
]
}
example: ./test/custom-istanbul-reporter.js, see istanbul built-in reporters' implementation for reference.
example: ./test/custom-v8-reporter.js
Multiple Reports:
const MCR = require('monocart-coverage-reports');
const coverageOptions = {
outputDir: './coverage-reports',
reports: [
['console-summary'],
['v8'],
['html', {
subdir: 'istanbul'
}],
['json', {
file: 'my-json-file.json'
}],
'lcovonly',
["custom-reporter-1"],
["custom-reporter-2", {
type: "istanbul",
key: "value"
}],
['/absolute/path/to/custom-reporter.js']
]
}
const mcr = MCR(coverageOptions);
Compare Reports
If the V8 data format is used for Istanbul reports, it will be automatically converted from V8 to Istanbul.
| Istanbul | V8 | V8 to Istanbul |
---|
Coverage data | Istanbul (Object) | V8 (Array) | V8 (Array) |
Output | Istanbul reports | V8 reports | Istanbul reports |
- Bytes | â | â
| â |
- Statements | â
| â
| â
|
- Branches | â
| â
| â
|
- Functions | â
| â
| â
|
- Lines | â
| â
| â
|
- Execution counts | â
| â
| â
|
CSS coverage | â | â
| â
|
Minified code | â | â
| â |
Collecting Istanbul Coverage Data
Collecting V8 Coverage Data
Collecting V8 Coverage Data with Playwright
await Promise.all([
page.coverage.startJSCoverage({
resetOnNavigation: false
}),
page.coverage.startCSSCoverage({
resetOnNavigation: false
})
]);
await page.goto("your page url");
const [jsCoverage, cssCoverage] = await Promise.all([
page.coverage.stopJSCoverage(),
page.coverage.stopCSSCoverage()
]);
const coverageData = [... jsCoverage, ... cssCoverage];
Collect coverage with @playwright/test
Automatic fixtures
, see example: fixtures.ts
For more examples, see ./test/test-v8.js, and anonymous, css
Collecting Raw V8 Coverage Data with Puppeteer
await Promise.all([
page.coverage.startJSCoverage({
resetOnNavigation: false,
includeRawScriptCoverage: true
}),
page.coverage.startCSSCoverage({
resetOnNavigation: false
})
]);
await page.goto("your page url");
const [jsCoverage, cssCoverage] = await Promise.all([
page.coverage.stopJSCoverage(),
page.coverage.stopCSSCoverage()
]);
const coverageData = [... jsCoverage.map((it) => {
return {
source: it.text,
... it.rawScriptCoverage
};
}), ... cssCoverage];
Example: ./test/test-puppeteer.js
Collecting V8 Coverage Data from Node.js
Possible solutions:
-
NODE_V8_COVERAGE=dir
- Sets Node.js env
NODE_V8_COVERAGE
=dir
before the program running, the coverage data will be saved to the dir
after the program exits gracefully. - Read the JSON file(s) from the
dir
and generate coverage report. - Example:
cross-env NODE_V8_COVERAGE=.temp/v8-coverage-env
node ./test/test-node-env.js && node ./test/generate-report.js
-
V8 API + NODE_V8_COVERAGE
- Writing the coverage started by NODE_V8_COVERAGE to disk on demand with
v8.takeCoverage()
, it does not require waiting until the program exits gracefully. - Example:
cross-env NODE_V8_COVERAGE=.temp/v8-coverage-api
node ./test/test-node-api.js
-
Inspector API
- Connecting to the V8 inspector and enable V8 coverage.
- Taking coverage data and adding it to the report.
- Example:
node ./test/test-node-ins.js
- vm Example (scriptOffset):
node ./test/test-node-vm.js
-
CDP API
- Enabling Node Debugging.
- Collecting coverage data with CDP API.
- Example:
node --inspect=9229 ./test/test-node-cdp.js
-
Node Debugging + CDP + NODE_V8_COVERAGE + V8 API
- When the program starts a server, it will not exit on its own, thus requiring a manual invocation of the
v8.takeCoverage()
interface to manually collect coverage data. Remote invocation of the v8.takeCoverage()
interface can be accomplished through the Runtime.evaluate
of the CDP. - Example for koa web server:
node ./test/test-node-koa.js
-
Child Process + NODE_V8_COVERAGE
Collecting V8 Coverage Data with CDPClient
API
startJSCoverage: () => Promise<void>;
stopJSCoverage: () => Promise<V8CoverageEntry[]>;
startCSSCoverage: () => Promise<void>;
stopCSSCoverage: () => Promise<V8CoverageEntry[]>;
startCoverage: () => Promise<void>;
stopCoverage: () => Promise<V8CoverageEntry[]>;
writeCoverage: () => Promise<string>;
getIstanbulCoverage: (coverageKey?: string) => Promise<any>;
- Work with node debugger port
--inspect=9229
or browser debugging port --remote-debugging-port=9229
const MCR = require('monocart-coverage-reports');
const client = await MCR.CDPClient({
port: 9229
});
await client.startJSCoverage();
const coverageData = await client.stopJSCoverage();
const { chromium } = require('playwright');
const MCR = require('monocart-coverage-reports');
const browser = await chromium.launch();
const page = await browser.newPage();
const session = await page.context().newCDPSession(page);
const client = await MCR.CDPClient({
session
});
await client.startCoverage();
await page.goto("your page url");
const coverageData = await client.stopCoverage();
const puppeteer = require('puppeteer');
const MCR = require('monocart-coverage-reports');
const browser = await puppeteer.launch({});
const page = await browser.newPage();
const session = await page.target().createCDPSession();
const client = await MCR.CDPClient({
session
});
await client.startCoverage();
await page.goto("your page url");
const coverageData = await client.stopCoverage();
const { Builder, Browser } = require('selenium-webdriver');
const MCR = require('monocart-coverage-reports');
const driver = await new Builder().forBrowser(Browser.CHROME).build();
const pageCdpConnection = await driver.createCDPConnection('page');
const session = new MCR.WSSession(pageCdpConnection._wsConnection);
const client = await MCR.CDPClient({
session
})
V8 Coverage Data API
export interface CoverageRange {
startOffset: integer;
endOffset: integer;
count: integer;
}
export interface FunctionCoverage {
functionName: string;
ranges: CoverageRange[];
isBlockCoverage: boolean;
}
export interface ScriptCoverage {
scriptId: Runtime.ScriptId;
url: string;
functions: FunctionCoverage[];
}
export type V8CoverageData = ScriptCoverage[];
JavaScript Runtime | V8 Coverage | |
---|
Chrome (65%) | â
| Chromium-based |
Safari (18%) | â | |
Edge (5%) | â
| Chromium-based |
Firefox (2%) | â | |
Node.js | â
| |
Deno | â | issue |
Bun | â | |
Filtering Results
Using entryFilter
and sourceFilter
to filter the results for V8 report
When V8 coverage data collected, it actually contains the data of all entry files, for example:
- dist/main.js
- dist/vendor.js
- dist/something-else.js
We can use entryFilter
to filter the entry files. For example, we should remove vendor.js
and something-else.js
if they are not in our coverage scope.
When inline or linked sourcemap exists to the entry file, the source files will be extracted from the sourcemap for the entry file, and the entry file will be removed if logging
is not debug
.
- src/index.js
- src/components/app.js
- node_modules/dependency/dist/dependency.js
We can use sourceFilter
to filter the source files. For example, we should remove dependency.js
if it is not in our coverage scope.
- src/index.js
- src/components/app.js
For example:
const coverageOptions = {
entryFilter: (entry) => entry.url.indexOf("main.js") !== -1,
sourceFilter: (sourcePath) => sourcePath.search(/src\//) !== -1
};
Or using minimatch
pattern:
const coverageOptions = {
entryFilter: "**/main.js",
sourceFilter: "**/src/**"
};
Support multiple patterns:
const coverageOptions = {
entryFilter: {
'**/node_modules/**': false,
'**/vendor.js': false,
'**/src/**': true
},
sourceFilter: {
'**/node_modules/**': false,
'**/**': true
}
};
As CLI args (JSON-like string. Added in: v2.8):
mcr --sourceFilter "{'**/node_modules/**':false,'**/**':true}"
Note, those patterns will be transformed to a function, and the order of the patterns will impact the results:
const coverageOptions = {
entryFilter: (entry) => {
if (minimatch(entry.url, '**/node_modules/**')) { return false; }
if (minimatch(entry.url, '**/vendor.js')) { return false; }
if (minimatch(entry.url, '**/src/**')) { return true; }
return false;
}
};
Using filter
instead of entryFilter
and sourceFilter
If you don't want to define both entryFilter
and sourceFilter
, you can use filter
instead. (Added in: v2.8)
const coverageOptions = {
filter: {
'**/node_modules/**': false,
'**/vendor.js': false,
'**/src/**': true
'**/**': true
}
};
Resolve sourcePath
for the Source Files
If the source file comes from the sourcemap, then its path is a virtual path. Using the sourcePath
option to resolve a custom path.
For example, we have tested multiple dist files, which contain some common files. We hope to merge the coverage of the same files, so we need to unify the sourcePath
in order to be able to merge the coverage data.
const coverageOptions = {
sourcePath: (filePath) => {
const list = ['my-dist-file1/', 'my-dist-file2/'];
for (const str of list) {
if (filePath.startsWith(str)) {
return filePath.slice(str.length);
}
}
return filePath;
}
};
It also supports simple key/value replacement:
const coverageOptions = {
sourcePath: {
'my-dist-file1/': '',
'my-dist-file2/': ''
}
};
Normalize the full path of the file:
const path = require("path")
const coverageOptions = {
sourcePath: (filePath, info)=> {
if (!filePath.includes('/') && info.distFile) {
return `${path.dirname(info.distFile)}/${filePath}`;
}
return filePath;
}
}
Adding Empty Coverage for Untested Files
By default the untested files will not be included in the coverage report, we can add empty coverage for untested files with option all
, the untested files will show 0% coverage.
const coverageOptions = {
all: './src',
all: ['./src', './lib'],
};
The untested files will apply to the sourceFilter
. And it also supports additional filter
(return the file type for js or css coverage):
const coverageOptions = {
all: {
dir: ['./src'],
filter: {
'**/ignored-*.js': false,
'**/*.html': false,
'**/*.scss': "css",
'**/*': true
}
}
};
We can also compile these untested files, such as .ts, .jsx, or .vue, etc., so that they can be analyzed by the default AST parser, thus get more coverage metric data.
const path = require("path");
const swc = require("@swc/core");
const coverageOptions = {
all: {
dir: ['./src'],
transformer: async (entry) => {
const { code, map } = await swc.transform(entry.source, {
filename: path.basename(entry.url),
sourceMaps: true,
isModule: true,
jsc: {
parser: {
syntax: "typescript",
jsx: true
},
transform: {}
}
});
entry.source = code;
entry.sourceMap = JSON.parse(map);
}
}
};
onEnd Hook
For example, checking thresholds:
const EC = require('eight-colors');
const coverageOptions = {
name: 'My Coverage Report',
outputDir: './coverage-reports',
onEnd: (coverageResults) => {
const thresholds = {
bytes: 80,
lines: 60
};
console.log('check thresholds ...', thresholds);
const errors = [];
const { summary } = coverageResults;
Object.keys(thresholds).forEach((k) => {
const pct = summary[k].pct;
if (pct < thresholds[k]) {
errors.push(`Coverage threshold for ${k} (${pct} %) not met: ${thresholds[k]} %`);
}
});
if (errors.length) {
const errMsg = errors.join('\n');
console.log(EC.red(errMsg));
}
}
}
Ignoring Uncovered Codes
To ignore codes, use the special comment which starts with v8 ignore
:
function uncovered() {
}
- Ignoring the next line or next N lines
const os = platform === 'wind32' ? 'Windows' : 'Other';
const os = platform === 'wind32' ? 'Windows' : 'Other';
if (platform === 'linux') {
console.log('hello linux');
}
Multiprocessing Support
The data will be added to [outputDir]/.cache
, After the generation of the report, this data will be removed unless debugging has been enabled or a raw report has been used, see Debug for Coverage and Sourcemap
- Main process, before the start of testing
const MCR = require('monocart-coverage-reports');
const coverageOptions = require('path-to/same-options.js');
const mcr = MCR(coverageOptions);
mcr.cleanCache();
- Sub process 1, testing stage 1
const MCR = require('monocart-coverage-reports');
const coverageOptions = require('path-to/same-options.js');
const mcr = MCR(coverageOptions);
await mcr.add(coverageData1);
- Sub process 2, testing stage 2
const MCR = require('monocart-coverage-reports');
const coverageOptions = require('path-to/same-options.js');
const mcr = MCR(coverageOptions);
await mcr.add(coverageData2);
- Main process, after the completion of testing
const MCR = require('monocart-coverage-reports');
const coverageOptions = require('path-to/same-options.js');
const mcr = MCR(coverageOptions);
await mcr.generate();
Command Line
The CLI will run the program as a child process with NODE_V8_COVERAGE=dir
until it exits gracefully, and generate the coverage report with the coverage data from the dir
.
npm i monocart-coverage-reports -g
mcr node ./test/specs/node.test.js -r v8,console-details --lcov
npm i monocart-coverage-reports
npx mcr node ./test/specs/node.test.js -r v8,console-details --lcov
mcr -c mcr.config.js -- sub-cli -c sub-cli.config.js
Config File
Loading config file by priority:
- Custom config file:
- CLI:
mcr --config <my-config-file-path>
- API:
await mcr.loadConfig("my-config-file-path")
mcr.config.js
mcr.config.cjs
mcr.config.mjs
mcr.config.json
- json formatmcr.config.ts
(requires preloading the ts execution module)
Merge Coverage Reports
The following usage scenarios may require merging coverage reports:
- When the code is executed in different environments, like Node.js
server side
and browser client side
(Next.js
for instance). Each environment may generate its own coverage report. Merging them can give a more comprehensive view of the test coverage. - When the code is subjected to different kinds of testing. For example,
unit tests
with Jest
might cover certain parts of the code, while end-to-end tests
with Playwright
might cover other parts. Merging these different coverage reports can provide a holistic view of what code has been tested. - When tests are run on different machines or containers, each might produce its own coverage report. Merging these can give a complete picture of the test coverage across all machines or shards.
Automatic Merging
- The
MCR
will automatically merge all the added coverage data when executing generate()
. And it supports adding coverage data asynchronously across processes, see Multiprocessing Support - For
Next.js
, it can actually add coverage data including both server side and client side before executing generate()
, see example nextjs-with-playwright - Using
Codecov
, a popular online code coverage service, which supports automatic merging of reports. Please use report codecov
, it will generate report file codecov.json
. If multiple codecov.json
files are generated, upload all these files, they will be automatically merged. see Codecov and merging reports
Manual Merging
If the reports cannot be merged automatically, then here is how to manually merge the reports.
First, using the raw
report to export the original coverage data to the specified directory.
- For example, we have
raw
coverage data from unit test
, which is output to ./coverage-reports/unit/raw
. Unit test examples:
const coverageOptions = {
name: 'My Unit Test Coverage Report',
outputDir: "./coverage-reports/unit",
reports: [
['raw', {
outputDir: "raw"
}],
['v8'],
['console-details']
]
};
-
We also have raw
coverage data from e2e test
, which is output to ./coverage-reports/e2e/raw
. E2E test examples:
-
Then create a script merge-coverage.js
to generate a merged report with option inputDir
.
const fs = require('fs');
const { CoverageReport } = require('monocart-coverage-reports');
const inputDir = [
'./coverage-reports/unit/raw',
'./coverage-reports/e2e/raw'
];
const coverageOptions = {
name: 'My Merged Coverage Report',
inputDir,
outputDir: './coverage-reports/merged',
entryFilter: {
'**/node_modules/**': false,
'**/*': true
},
sourceFilter: {
'**/node_modules/**': false,
'**/src/**': true
},
sourcePath: (filePath, info) => {
return filePath;
},
reports: [
['v8'],
['console-details']
],
onEnd: () => {
}
};
await new CoverageReport(coverageOptions).generate();
- Running script
node path/to/merge-coverage.js
after all the tests are completed. All the command scripts are probably like following:
{
"scripts": {
"test:unit": "jest",
"test:e2e": "playwright test",
"merge-coverage": "node path/to/merge-coverage.js",
"test": "npm run test:unit && npm run test:e2e && npm run merge-coverage"
}
}
see example: merge-code-coverage
Common issues
Unexpected coverage
In most cases, it happens when the coverage of the generated code is converted to the coverage of the original code through a sourcemap. In other words, it's an issue with the sourcemap. Most of the time, we can solve this by setting minify
to false
in the configuration of build tools. Let's take a look at an example:
const a = tf ? 'true' : 'false';
^ ^ ^
m1 p m2
In the generated code, there is a position p
, and we need to find out its corresponding position in the original code. Unfortunately, there is no matched mapping for the position p
. Instead, it has two adjacent upstream and downstream mappings m1
and m2
, so, the original position of p
that we are looking for, might not be able to be precisely located. Especially, the generated code is different from the original code, such as the code was minified, compressed or converted, it is difficult to find the exact original position without matched mapping.
How MCR
Works:
- 1, Trying to fix the original position with string comparison and
diff-sequences
. However, for non-JS code, such as Vue template, JSX, etc., it might be hard to find a perfect solution. - 2, Finding all functions, statements and branches by parsing the source code AST. (There is a small issue is the V8 cannot provide effective branch coverage information for
AssignmentPattern
)
Unparsable source
It happens during the parsing of the source code into AST, if the source code is not in the standard ECMAScript. For example ts
, jsx
and so on. There is a option to fix it, which is to manually compile the source code for these files.
import * as fs from "fs";
import * as path from "path";
import { fileURLToPath } from "url";
import * as TsNode from 'ts-node';
const coverageOptions = {
onEntry: async (entry) => {
const filePath = fileURLToPath(entry.url)
const originalSource = fs.readFileSync(filePath).toString("utf-8");
const fileName = path.basename(filePath);
const tn = TsNode.create({});
const source = tn.compile(originalSource, fileName);
entry.fake = false;
entry.source = source;
}
}
JavaScript heap out of memory
When there are a lot of raw v8 coverage files to process, it may cause OOM. We can try the following Node.js options:
- run: npm run test:coverage
env:
NODE_OPTIONS: --max-old-space-size=8192
Debug for Coverage and Sourcemap
Sometimes, the coverage is not what we expect. The next step is to figure out why, and we can easily find out the answer step by step through debugging.
- Start debugging for v8 report with option
logging: 'debug'
const coverageOptions = {
logging: 'debug',
reports: [
['v8'],
['console-details']
]
};
When logging
is debug
, the raw report data will be preserved in [outputDir]/.cache
or [outputDir]/raw
if raw
report is used. And the dist file will be preserved in the V8 list, and by opening the browser's devtool, it makes data verification visualization effortless.
- Show time logs with env
MCR_LOG_TIME
process.env.MCR_LOG_TIME = true
Integration with Any Testing Framework
- API
- Collecting coverage data when any stage of the test is completed, and adding the coverage data to the coverage reporter.
await mcr.add(coverageData)
- Generating the coverage reports after the completion of all tests.
await mcr.generate()
- see Multiprocessing Support
- CLI
- Wrapping with any CLI.
mcr your-cli --your-arguments
- see Command line
Integration Examples
- c8 has integrated
MCR
as an experimental feature since v10.1.0
c8 --experimental-monocart --reporter=v8 --reporter=console-details node foo.js
mcr mocha ./test/**/*.js
cross-env NODE_OPTIONS="--import tsx" npx mcr tsx ./src/example.ts
cross-env NODE_OPTIONS="--import tsx" npx mcr mocha ./test/**/*.ts
mcr --import tsx tsx ./src/example.ts
mcr --import tsx mocha ./test/**/*.ts
cross-env NODE_OPTIONS="--loader ts-node/esm --no-warnings" npx mcr ts-node ./src/example.ts
cross-env NODE_OPTIONS="--loader ts-node/esm --no-warnings" npx mcr mocha ./test/**/*.ts
mcr ava
const coverageOptions = {
outputDir: "./coverage-reports",
reports: [
['codecov']
]
};
- name: Codecov
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./coverage-reports/codecov.json
const coverageOptions = {
outputDir: "./coverage-reports",
lcov: true
};
- name: Codacy Coverage Reporter
uses: codacy/codacy-coverage-reporter-action@v1
with:
project-token: ${{ secrets.CODACY_PROJECT_TOKEN }}
coverage-reports: ./docs/mcr/lcov.info
const coverageOptions = {
outputDir: "./coverage-reports",
lcov: true
};
- name: Coveralls
uses: coverallsapp/github-action@v2
with:
files: ./coverage-reports/lcov.info
- Using
lcov
report. Github actions example:
- name: Analyze with SonarCloud
uses: sonarsource/sonarcloud-github-action@master
env:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
with:
projectBaseDir: ./
args: >
-Dsonar.organization=cenfun
-Dsonar.projectKey=monocart-coverage-reports
-Dsonar.projectName=monocart-coverage-reports
-Dsonar.javascript.lcov.reportPaths=docs/mcr/lcov.info
-Dsonar.sources=lib
-Dsonar.tests=test
-Dsonar.exclusions=dist/*,packages/*
Contributing
- Node.js 20+
- VSCode (extensions: eslint/stylelint/vue)
npm install
npx playwright install --with-deps
npm run build
npm run test
npm run dev
- Refreshing
eol=lf
for snapshot of test (Windows)
git add . -u
git commit -m "Saving files before refreshing line endings"
npm run eol
Thanks