OpenSSF Scorecard

OpenSSF Scorecard (formerly Security Scorecards) is an automated tool from the Open Source Security Foundation that grades a GitHub or GitLab repository's security practices. It runs 18 heuristic checks, such as Branch-Protection and Dangerous-Workflow, scores each 0 to 10, and combines them into a risk-weighted aggregate. It measures how a project is maintained, not what its published packages contain.

The most important fact about OpenSSF Scorecard is what it reads. It downloads a snapshot of the repository's default branch and queries the host's API for settings like branch protection. The package you install comes from somewhere else: a registry tarball uploaded by whoever holds publish credentials, with nothing forcing it to match the repository Scorecard graded. Our position is that Scorecard is a good tool for maintainers hardening a repository and a weak signal for consumers deciding whether a package is safe. Use individual checks as evidence about process. Never use the aggregate as an install gate.

OpenSSF Scorecard is an open source Go program that runs a fixed set of security heuristics against one source repository. For file-based checks it fetches the repository archive at the default branch head, which is what the --commit flag defaults to (source). For everything else it calls the GitHub or GitLab API: branch rules, recent pull requests, check runs, and release assets.

There are three ways to get a result. The CLI scores any public repository with a GitHub token. The ossf/scorecard-action workflow scores your own repository on every push and, with publish_results: true, publishes the result. A weekly batch job covers everyone else: as of November 2026, the Scorecard README says it scans more than 1.3 million repositories every week and serves the results through a REST API at api.scorecard.dev. The same README says the BigQuery public dataset is no longer available, so guides that still point there are out of date.

The default run has 18 checks. Each check has a risk level, and the risk level sets its weight in the aggregate.

Two more checks, Webhooks and SBOM, are experimental and run only when the SCORECARD_EXPERIMENTAL environment variable is set. Since version 5, Scorecard also exposes the finer-grained probes behind each check, 48 of them as of November 2026 per the probe documentation. They can be selected with --probes and printed with --format=probe.

The project is candid about what this adds up to. "The checks themselves are heuristics; there are false positives and false negatives," the README says in its list of non-goals.

An OpenSSF Scorecard aggregate is a weighted mean of check scores, weighted by each check's risk level. Critical checks weigh 10, High 7.5, Medium 5, and Low 2.5. The whole calculation lives in one function, GetAggregateScore:

weights := map[string]float64{"Critical": 10, "High": 7.5, "Medium": 5, "Low": 2.5}
// Note: aggregate score changes depending on which checks are run.
for i := range r.Checks {
	// ...
	// This indicates an inconclusive score.
	if check.Score < checker.MinResultScore {
		continue
	}
	total += rs
	score += rs * float64(check.Score)
}
return score / total, nil

The continue is the detail that most explanations miss. A check that cannot reach a conclusion returns -1, and the aggregate drops it from both the numerator and the denominator. A missing check does not count as a zero. It simply stops existing for that repository.

Here is a real record from the weekly scan, fetched from the public API for debug, one of the most depended-upon packages on npm:

curl -s https://api.scorecard.dev/projects/github.com/debug-js/debug

{
  "date": "2026-10-19",
  "repo": { "name": "github.com/debug-js/debug", "commit": "666dbc50680933bde9e7fde3a6c3241102e8f60b" },
  "scorecard": { "version": "v5.6.0", "commit": "61e84743356275f6ffca23b1b2f2cb945e19eb96" },
  "score": 2.6,
  "checks": [
    { "name": "Packaging", "score": -1, "reason": "packaging workflow not detected" },
    { "name": "Pinned-Dependencies", "score": -1, "reason": "no dependencies found" },
    { "name": "Maintained", "score": 0, "reason": "1 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0" },
    { "name": "Dangerous-Workflow", "score": -1, "reason": "no workflows found" },
    { "name": "Code-Review", "score": 4, "reason": "Found 12/30 approved changesets -- score normalized to 4" },
    { "name": "Token-Permissions", "score": -1, "reason": "No tokens found" },
    { "name": "Binary-Artifacts", "score": 10, "reason": "no binaries found in the repo" },
    { "name": "CII-Best-Practices", "score": 0, "reason": "no effort to earn an OpenSSF best practices badge detected" },
    { "name": "Security-Policy", "score": 0, "reason": "security policy file not detected" },
    { "name": "Fuzzing", "score": 0, "reason": "project is not fuzzed" },
    { "name": "License", "score": 10, "reason": "license file detected" },
    { "name": "Signed-Releases", "score": -1, "reason": "no releases found" },
    { "name": "Branch-Protection", "score": 0, "reason": "branch protection not enabled on development/release branches" },
    { "name": "SAST", "score": 0, "reason": "SAST tool is not run on all commits -- score normalized to 0" }
  ]
}

Each check object in the real response also carries details and documentation fields, removed here for length.

The figure reproduces the 2.6 by hand. Nine checks returned a score. Binary-Artifacts contributes 7.5 × 10 = 75, Code-Review 7.5 × 4 = 30, and License 2.5 × 10 = 25, for 130 points. Maintained, Branch-Protection, Security-Policy, Fuzzing, SAST, and CII-Best-Practices all scored 0 but still add their weight, for a denominator of 50. 130 ÷ 50 = 2.6. Five checks returned -1 and were dropped: Dangerous-Workflow (weight 10), Token-Permissions (7.5), Signed-Releases (7.5), Pinned-Dependencies (5), and Packaging (5). Their reasons were "no workflows found", "No tokens found", "no releases found", "no dependencies found", and "packaging workflow not detected". Had those five counted as zeros, the aggregate would have been 130 ÷ 85 = 1.5.

The consequence is counterintuitive. A repository with no CI workflows at all is never scored on Dangerous-Workflow, the only Critical check, or on Token-Permissions. A repository that runs workflows is. On the same API, chalk/chalk scored 4.1 on October 26, 2026, and one of its zeros is Token-Permissions: "detected GitHub workflow tokens with excessive permissions." Adding automation exposes a project to checks that a bare repository skips.

Individual checks score in three different styles. Maintained is proportional: commits plus maintainer issue activity in the last 90 days, divided by a target of 12 and capped at 10, using integer math (source). debug's "1 commit(s) and 0 issue activity" rounds down to 0. Code-Review is leveled over the most recent ~30 changesets: one unreviewed human change costs 7 points, several cost 10, and unreviewed bot changes cost 3. Branch-Protection is tiered, so points in a higher tier only count once every lower tier is satisfied.

Two OpenSSF Scorecard scores for different repositories can be computed over different sets of checks. The README says weekly-scan results omit CI-Tests, Contributors, and Dependency-Update-Tool because those checks are expensive to run at scale. The weekly records we pulled in October 2026 for debug-js/debug, chalk/chalk, and nrwl/nx each contain 14 checks, and none of them includes Vulnerabilities, the check that queries OSV for known vulnerabilities. The record for ossf/scorecard itself, dated October 31, 2026, contains all 18 checks and scores 8.6; it includes the three checks the weekly scan skips, so it comes from the project's own Scorecard Action. The 7.9 that the weekly scan gave nrwl/nx on October 26, 2026 and an 8.6 from a project's own Action are not on the same scale.

A high OpenSSF Scorecard score cannot tell you whether the package version you are about to install matches the repository it graded. Scorecard never downloads the npm, PyPI, or crates.io artifact. Its --npm and --pypi flags exist only to look up the source repository; the README says these flags "do not change the final evaluation for the checks."

The figure has two lanes. The left lane is what Scorecard reads: the debug-js/debug repository's default branch and host settings, which the weekly scan scored 2.6 in October 2026. The right lane is what npm install runs: whatever tarball was last published to registry.npmjs.org with valid credentials. It illustrates the gap with the Qix compromise. On September 8, 2025, an attacker phished the npm account of the maintainer known as Qix with a fake 2FA-reset email from the look-alike domain npmjs[.]help. The attacker then published malicious versions of chalk, debug, and more than a dozen related packages. Socket reported that these packages "generally receive 2-3 billion downloads per week." The payload hooked fetch, XMLHttpRequest, and browser wallet APIs to swap cryptocurrency addresses. The left lane never sees any of it, because publishing to npm does not require a commit, a tag, or a pull request. The attacker needed only the npm account, and the npm account is not among Scorecard's inputs.

The registry still records the window. The malicious versions are gone, but their publish timestamps remain in the packument:

curl -s https://registry.npmjs.org/debug | jq '{
  published: (.time | {"4.4.1", "4.4.2", "4.4.3"}),
  installable: [.versions | keys[] | select(startswith("4.4."))]
}'

{
  "published": {
    "4.4.1": "2025-05-13T20:56:36.431Z",
    "4.4.2": "2025-09-08T13:12:39.973Z",
    "4.4.3": "2025-09-13T17:25:19.732Z"
  },
  "installable": ["4.4.0", "4.4.1", "4.4.3"]
}

Version 4.4.2 has a publish time and no installable entry. For chalk, the malicious 5.6.1 was published at 13:13:05 UTC and the clean 5.6.2 at 14:47:54 UTC the same day. None of debug's remaining 4.4.x versions carries an npm provenance attestation. The only link from those tarballs to a commit is the gitHead field, which the publishing client writes and nobody verifies.

Account security is outside Scorecard's view by design. The README explains that two-factor authentication "is not a Scorecard check because GitHub and GitLab do not make that data about user accounts public." Registry accounts are no more visible, and a maintainer account takeover is precisely an attack on them.

The gap is not specific to npm. The backdoor in the backdoored xz release lived partly in a build script that existed only in the release tarballs. Andres Freund's original disclosure says the injected line "is not in the upstream source of build-to-host, nor is build-to-host used by xz in git." A tool that grades git cannot see a file that was never committed.

Even inside the repository, OpenSSF Scorecard sees less than its check names suggest.

Dangerous-Workflow, Token-Permissions, and Pinned-Dependencies analyze files from one snapshot of one branch. GitHub Actions does not work that way. A pull_request_target workflow runs the definition from whichever base branch the pull request targets, so a vulnerable workflow on any long-lived branch is reachable.

The Nx compromise turned on exactly that. A PR-title validation workflow that interpolated ${{ github.event.pull_request.title }} into a shell step landed on Nx's default branch on August 21, 2025 and was reverted on August 22. The attacker exploited it through outdated branches that still carried the workflow, stole an npm token on August 24, and published malicious Nx versions on August 26. GitHub Security Lab's follow-up advisory, GHSL-2025-111, found the same workflow on the 21.4.x and website-21 branches after the main fix. "Attackers may exploit any existing branch with a copy of the workflow," the advisory says. A Scorecard run against the default branch after August 22 would not have seen that workflow at all.

Signed-Releases looks at assets on the project's recent GitHub releases (the documentation cites the last five) for files ending in .asc, .sig, .minisig, .sigstore, .sigstore.json, or .intoto.jsonl. A signature file earns 8; a SLSA provenance file earns 10. The check documentation is explicit: "The check does not verify the signatures." Signed-Releases also never looks at registry attestations, so a package that publishes npm provenance but attaches nothing to GitHub releases gets "no releases found" and a -1. It carries a High weight of 7.5, more than Pinned-Dependencies at Medium, 5.

Pinned-Dependencies rewards hash pins in workflows, Dockerfiles, and shell scripts. Since GitHub introduced immutable releases, the check also treats a uses: reference to a tag as pinned when that tag is backed by an immutable release. That is a correct statement about mutability and a weak statement about safety.

The Trivy GitHub Actions compromise in March 2026 shows why. An attacker holding leaked credentials force-pushed 75 of the 76 version tags of aquasecurity/trivy-action to commits carrying an infostealer, and the compromised releases displayed GitHub's "Immutable" badge. In September 2026, two GitHub Actions compromised in the May 2026 Mini Shai-Hulud campaign, actions-cool/issues-helper and actions-cool/maintain-one-comment, came back online in the actions-cool re-enablement, with their malicious tags intact. GitHub's dependency graph lists about 15,000 dependent repositories for issues-helper alone. A SHA pin to a commit from before May 18 was safe. A SHA pin to a poisoned commit would have been just as frozen and just as malicious. Scorecard checks that you pinned. It cannot check what you pinned to.

Several checks reward the presence of a config file. Dependency-Update-Tool passes if a recognized update bot is configured, and its documentation says it "does not ensure that the tool is run or that the tool's pull requests are merged." It also does not ask whether those automated dependency updates wait out a dependency cooldown. An update bot that opens a pull request minutes after a hijacked release is the fastest route for that release into your lockfile, and it scores the same as one that waits a week. SAST passes when a recognized static analysis tool ran on recent merged pull requests, not on what it found or whether anyone fixed it.

OpenSSF Scorecard and package-level analysis answer different questions, and the useful comparison is about the unit each one inspects.

Socket's package scores sit in the third column. They are built from alerts raised on each published version, such as install scripts, obfuscated code, network access, and known malware. Per Socket's scoring documentation, any critical alert caps the affected category at 0.25 before scaling, however healthy the other signals look. Those scores are heuristics too, and the same page warns that its metrics may change. Some inputs, such as commit and issue counts, come from repository activity, so those parts share Scorecard's blind spots.

The verdict: use OpenSSF Scorecard to judge and improve how a repository is run, and use artifact-level package risk assessment to decide whether a specific version is safe to install. The OpenSSF's newer OSPS Baseline takes a third route: tiered controls that projects self-attest to, which are not meant to be used as a score.

For maintainers, the Scorecard Action is a cheap way to find real attack paths in your own repository. Fix Dangerous-Workflow first, because the Nx compromise ran through exactly that pattern. Then set top-level workflow permissions to read-only for Token-Permissions and pin third-party actions by full commit SHA to commits you have reviewed. After that, check every long-lived branch for the same workflow files, because Scorecard will not.

For consumers, read individual checks and probes instead of the aggregate. A Maintained score of 0 on a utility library is a prompt to read the code, not a verdict. An archived probe outcome is a fact. Nx's 7.9 is an average over whichever checks happened to run. And remember that the Code-Review and Branch-Protection checks largely measure how many people a project has. Solo maintainers fail them by construction, while the packages they ship may be perfectly sound.

The open problem is binding the grade to the artifact. Package provenance is the missing link: an npm or PyPI attestation that names the repository, commit, and workflow that built a tarball would make a Scorecard of that repository mean something about the package. But provenance is necessary, not sufficient. A compromised maintainer or a hijacked workflow publishes through the legitimate pipeline and gets a valid attestation. Until registries require that link and someone inspects what crosses it, an OpenSSF Scorecard score describes a repository you may not be running.