vlt Launches "reproduce": A New Tool Challenging the Limits of Package Provenance

vlt, an open-source JavaScript package manager positioned as a high-performance alternative to npm and yarn, has launched "reproduce," a tool that independently verifies whether published npm packages can be faithfully rebuilt from their source code.

Known for its improved dependency management features and serverless package registry, vlt continues to expand its security-focused tooling with this latest launch. The timing is significant as several high-profile supply chain attacks have recently demonstrated the limitations of existing provenance mechanisms.

What reproduce Does#

Unlike traditional provenance systems that associate packages with build environments, reproduce takes a more direct approach:

1. It fetches the package's source metadata from the npm registry
2. Clones the source repository at the exact commit linked to the package
3. Runs the build steps in a clean environment
4. Compares the resulting artifact with the actual published npm package

The tool offers notable features including backwards compatibility (this is significant because it means the tool can verify packages that were published before provenance was even available), manifest confusion detection (flagging mismatching repository information), and a simple CLI interface that returns clear exit codes for easy integration into CI/CD pipelines.

According to vlt's Darcy Clarke, when testing 5,000 high-impact npm packages (those that have either have >1 million weekly downloads or >500 dependents), they found that 5.78% were reproducible, compared to only 3.72% that had added provenance attestations after almost two years of availability. The vlt team was able to demonstrate that reproduce is already outperforming provenance adoption.

Recent Supply Chain Attacks Show Why Reproducibility Matters#

The launch of reproduce is particularly relevant given recent supply chain attacks that have exposed vulnerabilities in existing security measures:

In December 2024, the Ultralytics PyPI package was compromised four times in one weekend through GitHub Actions cache poisoning. Despite having valid provenance signatures, attackers were able to inject cryptocurrency mining malware into the package. Security researcher Adnan Khan noted at the time that "cache poisoning is unique in that it is very hard to detect for someone consuming an artifact."

Earlier this month, Socket researchers discovered a malicious typosquat of the BoltDB package in the Go ecosystem that exploited Go Module Proxy caching. After the malicious package was cached, the threat actor altered the Git tag to point to clean code. This meant developers inspecting the GitHub repository saw no malicious code, while those installing via the Go CLI received the compromised version.

Current Limitations and Future Plans#

While reproduce is innovative in its approach to package verification, it currently has limitations with complex build environments. The tool presently supports only one strategy (npm) and is designed to work with common build patterns rather than every possible configuration. As the documentation states, 'one-off/bespoke or complex configurations will not be supported,' though they plan to add more strategies as they find common patterns.

The tool executes build scripts from source repositories during verification, which could present risks without proper sandboxing—a feature the vlt team is likely to address in future updates.

For instance, a sophisticated attacker aware of this verification method could potentially target users by publishing a legitimate package while configuring its metadata to point to a commit containing malicious build scripts. This highlights the importance of running verification tools in isolated environments.

Like any tool that executes code during verification, reproduce has important security considerations. Clarke acknowledges this, noting: "We'll likely fast-follow this with a GHA to spin up the envs for you & run the test (making it easy to quasi-'sandbox'). You'll want to control your own env if you're truly security conscious and don't trust someone else's CI/config to lock down the env."

He clarified that in scenarios where malicious code exists in git references but not in the published package, the "reproducible check shouldn't pass if building+packing fails to produce the same integrity as exists in the registry." This integrity check is a key safeguard in the reproduce approach.

It's worth noting that npm provenance and PyPI provenance take complementary approaches by linking to specific build instructions rather than just git repositories. This provides additional transparency about the exact build process and offers another layer of verification. These different approaches to package verification—reproducibility and provenance—can work together to create a more secure ecosystem. In fact, reproducibility could be done using the exact build process recorded by the provenance data.

However, vlt indicates this is just the beginning. "If the git information you have been publishing with your packages is accurate, it is highly likely we will eventually define a strategy which maps to how you built your package from source," Clarke said. The company is working on new registry infrastructure and tooling that will incorporate more comprehensive reproducibility checks.

The reproduce tool is available at npmjs.com/reproduce or npmjs.com/vltpkg/reproduce.