a4bdb1649a
This PR adds Node24 support **Node/V8 changes:** - Move to new V8 syntax for some constructs (e.g. Cast<>) - Tracked down two issues which caused the read-only snapshot to be different across platforms: - in `read-only-serializer:258` the "live" pointer gets overwritten by the encoded slot (EncodedTagged). If V8_COMPRESS_POINTERS is not enabled, EncodedTagged is 4 bytes, but the live pointer is 8 bytes. memcpy() only copies EncodeTagged leaving 4bytes of "garbage" (=previous upper pointer half) which is semi-random per run - fixed this by clearing the whole value first - the CSA_HOLE_SECURITY_CHECK macro somehow leaves a string with a platform-dependent filename in the readonly heap as a constant, replaced that with "placeholder" - add PKG_TRACE_READONLY_HEAP blocks for debugging further snapshot issues Those fixes partially fix the sourceless cross-build issues introduced in Node 22 - For my test builds linux/linuxstatic/alpine/win on x64 are now compatible again. The heap contents on macos builds and other arm64 is wildly different, not sure why. I don't think this kind of compatibility is a goal for the V8 snapshot implementation in general. (This affects Node SEA as well, but SEA always has the source to fall back onto, which incurs a small performance hit) **Build Changes:** - Update linux/linuxcross build to use gcc 12 (required for Node 24) - For aloine/linuxstatic: Node 24 doesn't build on the muslcc image used previously, the image is unmaintained (although the toolchain builder isn't, we could build our own image) - Moved alpine/linuxstatic to native Alpine x64/arm64 toolchains, which seems to work quite well - Some minor upgrades to the Windows build, but had to enable `full-icu` for now because the ICU code gen step crashes with `small-icu` on the GH action build runners (not on my Windows test machine though) - Moved the macos build to macos 14 which is now required for Node 24 - x64 is now cross-built from arm64 because GH doesn't offer free macos14 x64 build workers - Removed all builds for Node 18 and older - **Notes:** - Needs more testing, I did some basic testing for x64 builds as well as macos/arm64 - I added a backport of the snapshot-determinism fixes to the latest Node 22 patch |
||
|---|---|---|
| .github | ||
| .vscode | ||
| lib | ||
| patches | ||
| scripts | ||
| .dockerignore | ||
| .eslintignore | ||
| .eslintrc | ||
| .gitignore | ||
| .prettierignore | ||
| CHANGELOG.md | ||
| DEVELOPMENT.md | ||
| Dockerfile.alpine | ||
| Dockerfile.alpine-muslcc | ||
| Dockerfile.linux | ||
| Dockerfile.linuxcross | ||
| license.md | ||
| package.json | ||
| README.md | ||
| shas.txt | ||
| tsconfig.json | ||
| yarn.lock | ||
A utility to fetch or build patched Node binaries used by pkg to generate executables. This repo hosts prebuilt binaries in Releases.
Binary Compatibility
| Node | Platform | Architectures | Minimum OS version |
|---|---|---|---|
| 81, 101, 121, 141, 161, 18, 20, 22 | alpine | x64, arm64 | 3.7.3, other distros with musl libc >= 1.1.18 |
| 81, 101, 121, 141, 161 | linux | x64 | Enterprise Linux 7, Ubuntu 14.04, Debian jessie, other distros with glibc >= 2.17 |
| 18, 20, 22 | linux | x64 | Enterprise Linux 8, Ubuntu 20.04, Debian buster, other distros with glibc >= 2.28 |
| 81, 101, 121, 141, 161 | linux | arm64 | Enterprise Linux 8, Ubuntu 18.04, Debian buster, other distros with glibc >= 2.27 |
| 18, 20, 22 | linux | arm64 | Enterprise Linux 9, Ubuntu 20.04, Debian bullseye, other distros with glibc >= 2.31 |
| 81, 101, 121, 141, 161, 18, 20, 22 | linuxstatic | x64, arm64 | Any distro with Linux Kernel >= 2.6.32 (>= 3.10 strongly recommended) |
| 161, 18, 20, 22 | linuxstatic | armv72 | Any distro with Linux Kernel >= 2.6.32 (>= 3.10 strongly recommended) |
| 81, 101, 121, 141, 161, 18, 20, 22 | macos | x64 | 10.13 |
| 141, 161, 18, 20, 22 | macos | arm643 | 11.0 |
| 81, 101, 121, 141, 161, 18, 20, 22 | win | x64 | 8.1 |
| 141, 161, 18, 20, 22 | win | arm64 | 10 |
[1]: end-of-life, may be removed in the next major release.
[2]: best-effort basis, not semver-protected.
[3]: mandatory code signing is enforced by Apple.
Security
We do not expect this project to have vulnerabilities of its own. Nonetheless, as this project distributes prebuilt Node.js binaries,
Node.js security vulnerabilities affect binaries distributed by this project, as well.
Like most of you, this project does not have access to advance/private disclosures of Node.js security vulnerabilities. We can only closely monitor the public security advisories from the Node.js team. It takes time to build and release a new set of binaries, once a new Node.js version has been released.
It is possible for this project to fall victim to a supply chain attack.
This project deploys multiple defense measures to ensure that the safe binaries are delivered to users:
- Binaries are compiled by Github Actions
- Workflows and build logs are transparent and auditable.
- Artifacts are the source of truth. Even repository/organization administrators can't tamper them.
- Hashes of binaries are hardcoded in source
- Origins of the binaries are documented.
- Changes to the binaries are logged by VCS (Git) and are publicly visible.
pkg-fetchrejects the binary if it does not match the hardcoded hash.
- GPG-signed hashes are available in Releases
- Easy to spot a compromise.
pkg-fetchpackage on npm is strictly permission-controlled- Only @robertsLando can push new revisions to npm.
Contributing Updates to Patches
Example workflow for applying patches to a new version of Node.js (18.13.0)
-
Clone Node.js as a sibling to your current
pkg-fetchclonegit clone -b v18.13.0 --single-branch https://github.com/nodejs/node.gitcd node
-
Attempt to apply the closest patch (e.g. applying the existing patch for 18.12.1 when trying to generate a new patch for 18.13.0)
git apply ..\pkg-fetch\patches\node.v18.12.1.cpp.patch --reject
- If no rejects, great! you are ready to make your new patch file.
git add -Agit diff --staged --src-prefix=node/ --dst-prefix=node/ > ..\pkg-fetch\patches\node.v18.13.0.cpp.patch
- If rejects exist, resolve them yourself, and ensure all changes are saved, and repeat step 4 to export the patch file
Resolving Rejects
Usually when a patch is rejected, it's because the context around the changes
was refactored slightly since the last patched version. This is not usually
complicated to resolve, but requires a human to interpret the changes since the
last version pkg was patched against, compared with the version you wish to
create a patch for.
One method is to pull up the diff for the file where the rejects apply for the
changes between the last tag (e.g. v18.12.1 to use the previous example) and the
tag you want a patch for (e.g. v18.13.0 to use the previous example). Alongside
this, have the .rej file and go through each rejected hunk by hunk and use
your best judgement to determine how it should apply against the new tag.
Save you results, and export the overall git diff with the commands from the example above.
Checking that patches apply cleanly
The expectation is that a patch applies cleanly, with no delta or offsets from the source repo.
When making a change to a patch file, it is possible to apply that patch without building by running
yarn applyPatches --node-range node18
where the --node-range can be specified to apply patches for the version of
node for which you are updating patches. If unspecified, the latest node version
in patches.json will be used.
Ultimately, the patch should result in fully functional node binary, but the
applyPatches script can be used to quickly iterate just the application of
the patches you are updating without needing to wait for the full build to
complete.
Building a Binary Locally
You can use the yarn start script to build the binary locally, which is helpful
when updating patches to ensure functionality before pushing patch updates for
review.
For example:
yarn start --node-range node18 --arch x64 --output dist
Small ICU
Node.js binaries built by this project use the small-icu option to reduce the size of the binary. If you need a Node.js binary with full ICU support, you can build it yourself by setting the --with-intl=full-icu option. Small ICU is sufficient for most use cases, and it reduces the binary size by about 30 MB.
Environment
| Var | Description |
|---|---|
PKG_BUILD_PATH |
Directory to use to clone and build nodejs binaries. Default to system temporary directory |
PKG_CACHE_PATH |
Path to pkg-cache. Default to ~/.pkg-cache |
PKG_IGNORE_TAG |
Ignore tag folder when checking local binary path |
PKG_NODE_PATH |
Custom path to the local nodejs binary to use |
HTTPS_PROXY |
Optional HTTPS proxy to use when fetching binaries |
HTTP_PROXY |
Optional HTTP proxy to use when fetching binaries |
MAKE_JOB_COUNT |
Number of parallel jobs when building binaries (value passed to make -j option). Default to number of cpus |
CFLAGS |
Flags to use when invoking C compiler |
CXXFLAGS |
Flags to use when invoking C++ compiler |
STRIP |
Path to strip command. Default to strip |