@apple/hierarchical-confusion-matrix
Advanced tools
[](https://www.npmjs.com/package/@apple/hierarchical-confusion-matrix)
The confusion matrix, a ubiquitous visualization for helping people evaluate machine learning models, is a tabular layout that compares predicted class labels against actual class labels over all data instances. Neo is a visual analytics system that enables practitioners to flexibly author and interact with hierarchical and multi-output confusion matrices, visualize derived metrics, renormalize confusions, and share matrix specifications.
This code accompanies the research paper:
Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels
Jochen Görtler, Fred Hohman, Dominik Moritz, Kanit Wongsuphasawat, Donghao Ren, Rahul Nair, Marc Kirchner, Kayur Patel
ACM Conference on Human Factors in Computing Systems (CHI), 2022.
Paper, 
Live demo, 
Video, 
Video Preview, 
Code
You can embed our confusion matrix visualization into your own project. There are two ways to use it.
Install with npm install --save @apple/hierarchical-confusion-matrix or yarn add @apple/hierarchical-confusion-matrix.
Then you can import the module in your project
import confMat from '@apple/hierarchical-confusion-matrix';
const spec = {
classes: ['root'],
};
const confusions = [
{
actual: ['root:a'],
observed: ['root:a'],
count: 1,
},
{
actual: ['root:a'],
observed: ['root:b'],
count: 2,
},
{
actual: ['root:b'],
observed: ['root:a'],
count: 3,
},
{
actual: ['root:b'],
observed: ['root:b'],
count: 4,
},
];
confMat.embed('matContainer', spec, confusions);
If you prefer to load the compiled JavaScript directly, you have to compile it. To do this, run yarn install and copy the public/confMat.js into your project. Here is a simple example of a small confusion matrix:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf8" />
<meta name="viewport" content="width=device-width" />
<title>Neo: Hierarchical Confusion Matrix</title>
</head>
<body>
<div id="matContainer"></div>
<script src="confMat.js"></script>
<script>
const spec = {
classes: ['root'],
};
const confusions = [
{
actual: ['root:a'],
observed: ['root:a'],
count: 1,
},
{
actual: ['root:a'],
observed: ['root:b'],
count: 2,
},
{
actual: ['root:b'],
observed: ['root:a'],
count: 3,
},
{
actual: ['root:b'],
observed: ['root:b'],
count: 4,
},
];
confMat.embed('matContainer', spec, confusions);
</script>
</body>
</html>
You can find all the options that you can pass via the spec argument in src/specification.ts.
The different loaders can be found in src/loaders, which include loading data from json, csv, vega, and a synthetic example synth for testing.
The confusions for data with actual labels of fruit:lemon that are incorrectly predicted as fruit:apple, of which there are count 1 of them.
{
"actual": ["fruit:lemon"],
"observed": ["fruit:apple"],
"count": 1
}
The confusions for hierarchical data with actual labels of fruit:citrus:lemon that are incorrectly predicted as fruit:pome:apple, of which there are count 2 of them. Note : denotes hierarchies.
{
"actual": ["fruit:citrus:lemon"],
"observed": ["fruit:pome:apple"],
"count": 2
}
The confusions for multi-output data with actual labels of fruit:lemon,taste:sweet that are incorrectly predicted as fruit:apple,taste:sour, of which there are count 3 of them. Note , denotes multi-ouput labels.
{
"actual": ["fruit:lemon", "taste:sweet"],
"observed": ["fruit:apple", "taste:sour"],
"count": 3
}
The confusions for hierarchical and multi-output data with actual labels of fruit:citrus:lemon,taste:sweet,ripeness:ripe that are incorrectly predicted as fruit:pome:apple,taste:sour,ripeness:not-ripe, of which there are count 4 of them.
{
"actual": [
"fruit:citrus:lemon",
"taste:sweet",
"ripeness:ripe"
],
"observed": [
"fruit:pome:apple",
"taste:sour"
"ripeness:not-ripe"
],
"count": 4
}
See fruit.json for a complete example of confusions for a hierarchical fruit, taste, and ripeness classification model.
Build:
yarn install
yarn build
Test:
yarn test:unit
Dev Server:
yarn dev
Lint & Fix:
yarn lint
When making contributions, refer to the CONTRIBUTING guidelines and read the CODE OF CONDUCT.
To cite our paper, please use:
@inproceedings{goertler2022neo,
title={Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels},
author={Görtler, Jochen and Hohman, Fred and Moritz, Dominik and Wongsuphasawat, Kanit and Ren, Donghao and Nair, Rahul and Kirchner, Marc and Patel, Kayur},
booktitle={Proceedings of the SIGCHI Conference on Human Factors in Computing Systems},
year={2022},
organization={ACM},
doi={10.1145/3491102.3501823}
}
This code is released under the LICENSE terms.
FAQs
[](https://www.npmjs.com/package/@apple/hierarchical-confusion-matrix)
The npm package @apple/hierarchical-confusion-matrix receives a total of 4 weekly downloads. As such, @apple/hierarchical-confusion-matrix popularity was classified as not popular.
We found that @apple/hierarchical-confusion-matrix demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket researchers uncover a malicious npm package posing as a tool for detecting vulnerabilities in Etherium smart contracts.
Security News
Research
A supply chain attack on Rspack's npm packages injected cryptomining malware, potentially impacting thousands of developers.
Research
Security News
Socket researchers discovered a malware campaign on npm delivering the Skuld infostealer via typosquatted packages, exposing sensitive data.