Back to glossary

Code Review

Code review is the practice of having another engineer inspect a change before it merges, to catch defects, security flaws, and malicious code.

Also known as peer review, peer code review, secure code review, pull request review, PR review

Code review, also called peer review, is the practice of having engineers inspect a proposed change, usually a pull request diff, before it merges. It catches defects, design problems, and security flaws in first-party code. Its blind spot is third-party code: a dependency update arrives as a few lockfile lines, and the package code behind them goes unread.

We measured that gap on a routine upgrade. Bumping axios from 1.20.0 to 1.21.0 changes 10 lines in package.json and package-lock.json. The published package code changes by 6,363 lines across 39 files, and 4,816 of those lines are build output that exists in no axios git commit. Our position: a dependency update is a code change, but no human can review it at that scale. Dependency review has to be automated, and it has to inspect the published artifact rather than the repository the artifact claims to come from.

What is code review?

#

Code review is a merge gate: a change lands only after someone other than its author reads the diff and approves it. On GitHub the unit of review is a pull request, and branch protection rules can require one or more approvals before anything merges. Reviewers check correctness, design, readability, test coverage, and security bugs such as injection flaws or missing authorization checks.

The process works because the changes are small and the reviewer knows the code. Google's study of 9 million reviewed changes found that the median change modifies 24 lines. The median review at Google finishes in under 4 hours, and fewer than 25% of changes have more than one reviewer. One engineer reading 24 lines of a codebase they work in every day is a fair contest.

As a security control, code review rests on three assumptions:

  • The diff is the change. What the reviewer reads is what will run.
  • The author is acting in good faith, or the reviewer can tell when they are not.
  • Someone actually reads the diff.

Software supply chain attacks are, almost by definition, attacks on those three assumptions. The rest of this entry shows where each one breaks.

Why does code review miss dependency changes?

#

Code review misses dependency changes because a dependency update's diff contains a pointer to code, not the code itself. A lockfile entry records a package name, a version, a tarball URL, and an integrity hash. The reviewer approves the hash. The package manager then installs whatever the hash identifies.

These are the changed lines in package-lock.json for the axios upgrade, produced by npm 10.9.9 on November 6, 2026, with surrounding context trimmed:

Diff
@@ -9,7 +9,7 @@
       "dependencies": {
-        "axios": "1.20.0"
+        "axios": "1.21.0"
       }
@@ -31,9 +31,9 @@
     "node_modules/axios": {
-      "version": "1.20.0",
-      "resolved": "https://registry.npmjs.org/axios/-/axios-1.20.0.tgz",
-      "integrity": "sha512-r8aOh8j9cGKpgQAqpzrUHnSIc6a59Y3Xf/cv8sy1DrHCkZHzQGEuoq1tARk6qSyDdtQGSDgpb9kFlruzPvrgwg==",
+      "version": "1.21.0",
+      "resolved": "https://registry.npmjs.org/axios/-/axios-1.21.0.tgz",
+      "integrity": "sha512-YCzXMB53Q9PZBsh7yN7wtapit4WepwcHS2fP8QhGsBxSz5YtgYobV7GGlWlAJi+RxfU77GQTQi2DSLUma42EYA==",
       "license": "MIT",

To see what that hash points to, compare the two published tarballs. npm has a built-in command for it, npm diff, which fetches both versions from the registry and diffs their contents:

Bash
$ npm diff --diff=axios@1.20.0 --diff=axios@1.21.0 --diff-name-only | wc -l
39
$ npm diff --diff=axios@1.20.0 --diff=axios@1.21.0 \
    | grep -E '^[+-]' | grep -vE '^(\+\+\+|---)' | wc -l
6363

The figure draws both diffs on the same scale. The pull request changes 10 lines: 2 in package.json and 8 in package-lock.json. The tarball diff changes 6,363 lines across 39 files. Of those, 1,357 are in lib/, which matches the axios git repository. Another 4,816 are in dist/, the bundled and minified builds that axios excludes from git. The remaining 190 are in type definitions, the README, and package.json.

Two things widen the gap in practice. First, GitHub hides most lockfiles by default. Linguist classifies package-lock.json, pnpm-lock.yaml, Cargo.lock, poetry.lock, uv.lock, and composer.lock as generated files, and GitHub collapses generated files in diffs until someone loads them. Second, adding a dependency produces far more lockfile than upgrading one. Adding esbuild 0.25.0 to the same project was a one-line change to package.json and a 443-line lockfile diff with 26 new package entries. The line that matters most, "hasInstallScript": true, is line 423 of that diff.

Some dependency code never reaches a diff at all. During the axios compromise in March 2026, CI pipelines that ran command-line tools through npx resolved the malicious version at run time, with no lockfile and no pull request involved. That path is covered in the hidden blast radius of the axios compromise. Code review cannot gate code that never passes through a pull request.

The code you review is not the code you install

#

The first assumption, that the diff is the change, also fails one level up. A registry tarball is an independent artifact. The publisher builds it, packs it, and uploads it, and neither npm nor PyPI requires its contents to match any commit in any repository.

The axios 1.21.0 tarball shows how normal that is. Its lib/ directory matches the v1.21.0 git tag file for file; we diffed them. Its dist/ directory holds 26,998 lines of bundled JavaScript and source maps that the axios .gitignore excludes from the repository. axios 1.21.0 carries npm provenance, which links the tarball to a CI build from a specific commit. Still, no pull request in the axios repository ever displays that build output.

Attackers have used this gap for years. In event-stream in 2018, a new maintainer shipped a payload aimed at a Bitcoin wallet application through a dependency, flatmap-stream. The malicious code was in the minified file published to npm and not in the readable source. A commenter on the GitHub issue that exposed it put it plainly: "The attack was part of the minified code, not part of the visible codebase checked into the repository."

In the xz Utils backdoor in 2024, the split was deliberate. The obfuscated payload was committed to git as two binary test files, where a reviewer would see only opaque blobs. The code that extracted and activated it existed only in the release tarballs. Andres Freund, who discovered the backdoor, wrote: "One portion of the backdoor is solely in the distributed tarballs." The backdoored 5.6.0 and 5.6.1 tarballs were created and signed by Jia Tan, a contributor with release access.

In the axios compromise, there was no source to review at all. axios 1.14.1 and 0.30.4 were published to npm with no matching git tags. Their main change was a new dependency, plain-crypto-js@4.2.1, which ran a dropper from a postinstall script and then replaced its own package.json to hide the evidence. Our automated detection flagged it within six minutes of publication.

Checking whether a tarball matches its source is possible but rare. When vlt tested 5,000 high-impact npm packages with its reproduce tool in February 2025, only 5.78% could be rebuilt from source to match what was published. Only 3.72% had provenance attestations.

The figure traces five stages from upstream source to your machine. Stage 1, the upstream pull request, gets human review; the xz payload entered here as binary test files, and the 2026 AISI agent tried and failed to land a dropper here. Stage 2, the release build, gets no review; the xz activation code and the TanStack compromise entered here. Stage 3, the published tarball, gets no review; event-stream and axios 1.14.1 entered here. Stage 4, your lockfile pull request, gets review of a pointer only. Stage 5, install and run, gets no review, and npx or global installs reach it without passing through stage 4.

How do attackers get malicious code past code review?

#

Attackers get malicious code past code review in three ways: they make it look routine, they supply their own reviewer, or they route around review entirely.

Looking routine is the cheap version. In November 2024, GitHub removed malicious pull requests that an account named evildojo666 had sent to 27 repositories. One added a single obfuscated line to models.py under a reasonable title. "I could definitely see a dev quickly approving PR's missing something like this," said Alex Cheema, whose project was targeted.

Supplying the reviewer defeats the good-faith assumption. In a UK AI Security Institute evaluation in July 2026, an agent running Anthropic's Mythos 5 hid a malware dropper behind a legitimate bug fix in a real open source project. When challenged, it used a second GitHub identity to pose as an independent user who had reviewed the pull request and found it safe. The maintainer rejected the change anyway. In xz Utils, the attacker was the person cutting releases, so the review boundary sat on the wrong side of them.

Contributor reputation, the signal reviewers lean on hardest, is also getting cheaper to build. In February 2026, an undisclosed AI agent operating as Kai Gritun opened 103 pull requests across 95 repositories in two weeks, and several merged into major projects after human review. Those changes were not malicious. They show that a track record which once took an attacker years can now be manufactured in days.

Routing around review is the most reliable. A maintainer account takeover lets an attacker publish straight to the registry with no commit to review, as appears to have happened with axios. CI compromise achieves the same result with a cleaner audit trail. In the TanStack compromise in May 2026, attacker code reached the project's GitHub Actions runner through a pull_request_target workflow and cache poisoning. It extracted an OIDC token and published 84 malicious package artifacts through TanStack's trusted publishing setup. Packages published that way can carry valid provenance, which is why provenance is necessary but not sufficient.

How do you review a dependency update?

#

You review a dependency update by reading what a normal pull request hides: the full lockfile diff, the diff between published tarballs, and the behavior the new code adds. Each step catches something the others miss.

Read the lockfile for structure, not versions

#

A lockfile cannot show you code, but it does show structure. Four signals are worth checking in every dependency pull request:

  • New entries you did not ask for. A release of an HTTP client that suddenly adds a crypto package, as axios 1.14.1 did with plain-crypto-js, is the most visible tell a lockfile offers.
  • "hasInstallScript": true on a new or changed entry. npm records the flag for packages that declare install-time scripts.
  • A resolved URL outside your registry. Lockfile injection swaps a tarball URL in a pull request and counts on nobody expanding the file.
  • An integrity change with no version change. The same version of a package should never hash differently.

lockfile-lint checks the last two mechanically:

Bash
$ npx lockfile-lint --path package-lock.json --type npm \
    --allowed-hosts npm --validate-https \
    --validate-package-names --validate-integrity
 ✔ No issues detected

The lockfile-lint README is candid about the limit: allowing a registry host means trusting every package that host serves.

Read the tarball diff, not the GitHub compare view

#

For a dependency you care about, npm diff --diff=pkg@old --diff=pkg@new shows the code you will actually install, including build output that never appears on GitHub. Look for new imports of child_process, net, or dns; new reads of environment variables or credential files; eval or Function applied to decoded strings; long encoded strings; new binaries; and changes to the scripts field in package.json. This step does not scale. The axios upgrade alone was 6,363 changed lines, and adding esbuild pulled in 26 packages at once.

Decide which changes a human must see

#

Dependency bots make updating cheap, and they make skipping review cheap too. Automated dependency updates that merge on green CI turn the lockfile into a direct channel from the registry to production. Our analysis of the axios blast radius lists automatically merged dependency update pull requests among the cases where lockfile protection breaks down.

A dependency cooldown delays new versions for a few days so that scanners and other users find short-lived malicious releases first. Its cost is patch latency: a cooldown also delays the fix for a vulnerability that is being exploited now, a trade-off we discuss in The Code You Didn't Write Is Still Yours to Defend.

npm v12, released July 8, 2026, no longer runs dependency install scripts by default. That removes install time as an execution point for npm users. It does not stop a payload that runs when your code imports the package.

ApproachWhat it inspectsCatchesMisses
Reviewing the upstream repositorySource commits on GitHubMalicious commits that land in gitAnything added at build or publish time: event-stream, the xz activation code, axios 1.14.1
Reading the lockfile diffNames, versions, URLs, hashes, install-script flagsUnexpected new packages, foreign hosts, new install scriptsEverything inside the tarball
Reading the tarball diffThe published codePayloads in build output and minified filesObfuscated or binary payloads a human cannot read; does not scale past a few packages
Automated package analysisContents and behavior of every added or changed packageInstall scripts, network, shell, and environment access, obfuscation, known malwarePayloads written to look like ordinary code; alerts still need triage
Dependency cooldownNothing; it waitsMalicious releases removed within the windowSlow-burn backdoors; it also delays security patches

Verdict: humans should review first-party diffs and every decision to add a new dependency. Every dependency change, including transitive ones, should also go through automated analysis of the published artifact, with merges blocked on serious findings. A review of the upstream GitHub repository is not a review of the package. Neither is a CVE scan: none of the malicious releases in this entry had a CVE when it was published.

Socket for GitHub runs that analysis inside the pull request. It posts an Overview Comment that lists each added or changed direct dependency, plus an Alert Comment for findings your security policy marks as important. Blocking alerts prevent the merge until someone resolves them. The analysis reads the published package code, not the repository it claims to come from. It judges behavior, so a payload written to look like ordinary library code can still get through, and warning-level alerts only help if someone reads them.

Review the artifact, not the repository

#

Code review was built for the code you write, and for that code it remains one of the few controls that catches design mistakes and bugs before they ship. Its failure in the supply chain is not careless reviewers. The thing under review is a pointer, and the pointer always looks fine. Until registries can prove that a tarball was built from a reviewed commit, and vlt could confirm that for fewer than 6% of high-impact npm packages in early 2025, the honest unit of dependency review is the published artifact. The question in a dependency pull request is never whether the diff looks right. It is what the hash points to.