What is pkg-fetch?
The pkg-fetch npm package is used to fetch and cache Node.js binaries for the pkg project. It is primarily used to download Node.js binaries that are compatible with the pkg tool, which allows you to package Node.js applications into standalone executables.
What are pkg-fetch's main functionalities?
Fetch Node.js binaries
This feature allows you to fetch a specific version of Node.js binaries for a given platform and architecture. The code sample demonstrates how to fetch Node.js version 14.15.1 for a Linux x64 system.
const fetch = require('pkg-fetch');
(async () => {
const nodeVersion = 'node-v14.15.1';
const platform = 'linux';
const arch = 'x64';
const binaryPath = await fetch.need({ nodeRange: nodeVersion, platform, arch });
console.log(`Binary path: ${binaryPath}`);
})();
Cache management
This feature provides access to the cache path where the fetched binaries are stored. The code sample shows how to retrieve and print the cache path.
const fetch = require('pkg-fetch');
(async () => {
const cachePath = fetch.system.cache;
console.log(`Cache path: ${cachePath}`);
})();
Check for binary existence
This feature allows you to check if a specific Node.js binary already exists in the cache. The code sample demonstrates how to check for the existence of Node.js version 14.15.1 for a Linux x64 system.
const fetch = require('pkg-fetch');
(async () => {
const nodeVersion = 'node-v14.15.1';
const platform = 'linux';
const arch = 'x64';
const exists = await fetch.exists({ nodeRange: nodeVersion, platform, arch });
console.log(`Binary exists: ${exists}`);
})();
Other packages similar to pkg-fetch
nexe
Nexe is a command-line utility that compiles your Node.js application into a single executable file. It also fetches Node.js binaries but focuses more on the compilation and packaging of the application. Compared to pkg-fetch, nexe provides a more integrated solution for creating standalone executables.
pkg
Pkg is a tool that packages Node.js projects into executables for various platforms. It uses pkg-fetch internally to fetch the necessary Node.js binaries. While pkg-fetch focuses on fetching binaries, pkg provides a higher-level interface for packaging applications.
node-pre-gyp
Node-pre-gyp is a tool that makes it easy to publish and install Node.js C++ addons from binaries. It fetches pre-built binaries for Node.js modules, similar to how pkg-fetch fetches Node.js binaries. However, node-pre-gyp is more focused on native addons rather than standalone executables.
A utility to fetch or build patched Node binaries used by pkg to generate executables. This repo hosts prebuilt binaries in Releases.
Binary Compatibility
Node | Platform | Architectures | Minimum OS version |
---|
81, 101, 121, 14, 16, 18 | alpine | x64, arm64 | 3.7.3, other distros with musl libc >= 1.1.18 |
81, 101, 121, 14, 16, 18 | linux | x64 | Enterprise Linux 7, Ubuntu 14.04, Debian jessie, other distros with glibc >= 2.17 |
81, 101, 121, 14, 16, 18 | linux | arm64 | Enterprise Linux 8, Ubuntu 18.04, Debian buster, other distros with glibc >= 2.27 |
81, 101, 121, 14, 16, 18 | linuxstatic | x64, arm64 | Any distro with Linux Kernel >= 2.6.32 (>= 3.10 strongly recommended) |
16, 18 | linuxstatic | armv72 | Any distro with Linux Kernel >= 2.6.32 (>= 3.10 strongly recommended) |
81, 101, 121, 14, 16, 18 | macos | x64 | 10.13 |
14, 16, 18 | macos | arm643 | 11.0 |
81, 101, 121, 14, 16, 18 | win | x64 | 8.1 |
14, 16, 18 | win | arm64 | 10 |
[1]: end-of-life, may be removed in the next major release.
[2]: best-effort basis, not semver-protected.
[3]: mandatory code signing is enforced by Apple.
Security
We do not expect this project to have vulnerabilities of its own. Nonetheless, as this project distributes prebuilt Node.js binaries,
Node.js security vulnerabilities affect binaries distributed by this project, as well.
Like most of you, this project does not have access to advance/private disclosures of Node.js security vulnerabilities. We can only closely monitor the public security advisories from the Node.js team. It takes time to build and release a new set of binaries, once a new Node.js version has been released.
It is possible for this project to fall victim to a supply chain attack.
This project deploys multiple defense measures to ensure that the safe binaries are delivered to users:
- Binaries are compiled by Github Actions
- Workflows and build logs are transparent and auditable.
- Artifacts are the source of truth. Even repository/organization administrators can't tamper them.
- Hashes of binaries are hardcoded in source
- Origins of the binaries are documented.
- Changes to the binaries are logged by VCS (Git) and are publicly visible.
pkg-fetch
rejects the binary if it does not match the hardcoded hash.
- GPG-signed hashes are available in Releases
- Easy to spot a compromise.
pkg-fetch
package on npm is strictly permission-controlled
- Only authorized Vercel employees can push new revisions to npm.
Contributing Updates to Patches
Example workflow for applying patches to a new version of Node.js (18.13.0)
- Clone Node.js as a sibling to your current
pkg-fetch
clone
git clone https://github.com/nodejs/node.git
cd node
- Checkout the tag you wish to generate a patch for
- Attempt to apply the closest patch (e.g. applying the existing patch for
18.12.1 when trying to generate a new patch for 18.13.0)
git apply ..\pkg-fetch\patches\node.v18.12.1.cpp.patch --reject
- If no rejects, great! you are ready to make your new patch file.
git add -A
git diff --staged --src-prefix=node/ --dst-prefix=node/ > ..\pkg-fetch\patches\node.v18.13.0.cpp.patch
- If rejects exist, resolve them yourself, and ensure all changes are saved,
and repeat step 4 to export the patch file
Resolving Rejects
Usually when a patch is rejected, it's because the context around the changes
was refactored slightly since the last patched version. This is not usually
complicated to resolve, but requires a human to interpret the changes since the
last version pkg
was patched against, compared with the version you wish to
create a patch for.
One method is to pull up the diff for the file where the rejects apply for the
changes between the last tag (e.g. v18.12.1 to use the previous example) and the
tag you want a patch for (e.g. v18.13.0 to use the previous example). Alongside
this, have the .rej
file and go through each rejected hunk by hunk and use
your best judgement to determine how it should apply against the new tag.
Save you results, and export the overall git diff with the commands from the
example above.
Checking that patches apply cleanly
The expectation is that a patch applies cleanly, with no delta or offsets from
the source repo.
When making a change to a patch file, it is possible to apply that patch without
building by running
yarn applyPatches --node-range node18
where the --node-range
can be specified to apply patches for the version of
node for which you are updating patches. If unspecified, the latest node version
in patches.json will be used.
Ultimately, the patch should result in fully functional node binary, but the
applyPatches
script can be used to quickly iterate just the application of
the patches you are updating without needing to wait for the full build to
complete.
Building a Binary Locally
You can use the yarn start
script to build the binary locally, which is helpful
when updating patches to ensure functionality before pushing patch updates for
review.
For example:
yarn start --node-range node18 --arch x64 --output dist