forked from vercel/pkg
-
-
Notifications
You must be signed in to change notification settings - Fork 64
feat(sea): let users override the base Node binary #279
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
cstrahan
wants to merge
9
commits into
yao-pkg:main
Choose a base branch
from
cstrahan:cstrahan/sea-node-override
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 3 commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
408dfde
feat(sea): let users override the base Node binary
cstrahan 8dd9bc1
fix(sea): rework base-node override around PKG_NODE_PATH (review feed…
cstrahan 9d9d4d3
fix(sea): validate the custom base binary's platform/arch against the…
cstrahan cc6e680
fix(sea): replace binary sniffing with a single probeNode (review fee…
cstrahan f5a1d3a
fix(sea): correct pkg->Node platform/arch translation in the custom-b…
cstrahan f47221b
fix(sea): enforce the SEA Node floor for 'latest'-targeted custom bases
cstrahan 32b66d4
refactor(sea): have probeNode emit JSON instead of space-joined fields
cstrahan 1648fc2
Merge branch 'main' into cstrahan/sea-node-override
robertsLando a2834a7
Merge branch 'main' into cstrahan/sea-node-override
robertsLando File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -9,6 +9,7 @@ import { | |
| mkdtemp, | ||
| stat, | ||
| readFile, | ||
| open, | ||
| } from 'fs/promises'; | ||
| import { createWriteStream } from 'fs'; | ||
| import { pipeline } from 'stream/promises'; | ||
|
|
@@ -329,6 +330,201 @@ async function getNodeVersion( | |
| return latest.version as NodeVersion; | ||
| } | ||
|
|
||
| /** | ||
| * The custom base Node binary to embed for SEA, or `undefined` to download one. | ||
| * | ||
| * Precedence matches standard mode: an explicit `opts.nodePath` (from the | ||
| * `--sea-node-path` CLI flag or the `seaNodePath` pkg-config key) wins over the | ||
| * `PKG_NODE_PATH` environment variable. `PKG_NODE_PATH` is the same env var | ||
| * pkg-fetch honours for the traditional build path (`localPlace()`); folding it | ||
| * in here makes it work for SEA too, instead of a separate SEA-only mechanism. | ||
| */ | ||
| function resolveCustomBaseNode( | ||
| opts: GetNodejsExecutableOptions, | ||
| ): string | undefined { | ||
| if (opts.nodePath) return opts.nodePath; | ||
| if (process.env.PKG_NODE_PATH) return resolve(process.env.PKG_NODE_PATH); | ||
| return undefined; | ||
| } | ||
|
|
||
| /** Executable container format sniffed from a binary's magic bytes. */ | ||
| type BinaryFormat = 'elf' | 'macho' | 'pe'; | ||
|
|
||
| const ELF_MACHINE: Record<number, NodeArch> = { | ||
| 0x3e: 'x64', // EM_X86_64 | ||
| 0xb7: 'arm64', // EM_AARCH64 | ||
| }; | ||
| const MACHO_CPU: Record<number, NodeArch> = { | ||
| 0x01000007: 'x64', // CPU_TYPE_X86_64 | ||
| 0x0100000c: 'arm64', // CPU_TYPE_ARM64 | ||
| }; | ||
| const PE_MACHINE: Record<number, NodeArch> = { | ||
| 0x8664: 'x64', // IMAGE_FILE_MACHINE_AMD64 | ||
| 0xaa64: 'arm64', // IMAGE_FILE_MACHINE_ARM64 | ||
| }; | ||
|
|
||
| /** | ||
| * Expected container format for a pkg target platform (the raw suffix string). | ||
| * ELF covers the whole Linux/Alpine family (linux / alpine / linuxstatic / | ||
| * freebsd) — the glibc/musl/static flavor isn't in the header. | ||
| */ | ||
| const formatForPlatform = (platform: string): BinaryFormat => | ||
| platform === 'macos' || platform === 'darwin' | ||
| ? 'macho' | ||
| : platform === 'win' | ||
| ? 'pe' | ||
| : 'elf'; | ||
| const FORMAT_LABEL: Record<BinaryFormat, string> = { | ||
| elf: 'ELF (Linux/Alpine)', | ||
| macho: 'Mach-O (macOS)', | ||
| pe: 'PE (Windows)', | ||
| }; | ||
|
|
||
| /** | ||
| * Sniff a binary's container format and CPU arch from its magic bytes, so we can | ||
| * tell whether a supplied base Node binary actually matches the requested | ||
| * target. Reads only the header. Returns `{}` for an unrecognised file (caller | ||
| * skips the format/arch checks rather than guessing). Note: ELF can't reveal the | ||
| * glibc/musl/static *flavor*, so `elf` maps to the whole Linux/Alpine family. | ||
| */ | ||
| export async function sniffBinaryTarget( | ||
| file: string, | ||
| ): Promise<{ format?: BinaryFormat; arch?: NodeArch }> { | ||
| let fh; | ||
| try { | ||
| fh = await open(file, 'r'); | ||
| } catch { | ||
| return {}; | ||
| } | ||
| try { | ||
| const buf = Buffer.alloc(4096); | ||
| const { bytesRead } = await fh.read(buf, 0, 4096, 0); | ||
| const b = buf.subarray(0, bytesRead); | ||
| if (b.length < 20) return {}; | ||
|
|
||
| // ELF: 0x7f 'E' 'L' 'F'; EI_DATA at [5] (1=LE,2=BE); e_machine at [18..20]. | ||
| if (b[0] === 0x7f && b[1] === 0x45 && b[2] === 0x4c && b[3] === 0x46) { | ||
| const machine = b[5] === 2 ? b.readUInt16BE(18) : b.readUInt16LE(18); | ||
| return { format: 'elf', arch: ELF_MACHINE[machine] }; | ||
| } | ||
|
|
||
| // Mach-O (thin): magic FEEDFACE/FEEDFACF; byte-swapped CEFAEDFE/CFFAEDFE are | ||
| // the little-endian on-disk forms. cputype is the next 4 bytes. | ||
| const be = b.readUInt32BE(0); | ||
| if ( | ||
| be === 0xfeedface || | ||
| be === 0xfeedfacf || | ||
| be === 0xcefaedfe || | ||
| be === 0xcffaedfe | ||
| ) { | ||
| const le = be === 0xcefaedfe || be === 0xcffaedfe; | ||
| const cpu = le ? b.readUInt32LE(4) : b.readUInt32BE(4); | ||
| return { format: 'macho', arch: MACHO_CPU[cpu] }; | ||
| } | ||
|
|
||
| // PE: 'MZ', e_lfanew (uint32 @0x3C) -> 'PE\0\0', COFF machine 2 bytes after. | ||
| if (b[0] === 0x4d && b[1] === 0x5a && b.length >= 0x40) { | ||
| const lfanew = b.readUInt32LE(0x3c); | ||
| if ( | ||
| lfanew + 6 <= b.length && | ||
| b[lfanew] === 0x50 && | ||
| b[lfanew + 1] === 0x45 && | ||
| b[lfanew + 2] === 0 && | ||
| b[lfanew + 3] === 0 | ||
| ) { | ||
| return { format: 'pe', arch: PE_MACHINE[b.readUInt16LE(lfanew + 4)] }; | ||
| } | ||
| return { format: 'pe' }; | ||
| } | ||
|
|
||
| return {}; | ||
| } finally { | ||
| await fh.close(); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Guard a custom base Node binary against multi-target / wrong-platform / | ||
| * version-skew footguns. | ||
| * | ||
| * A supplied binary (via `--sea-node-path` / `seaNodePath` / `PKG_NODE_PATH`, or | ||
| * `useLocalNode`) is returned by {@link getNodejsExecutable} for *every* target, | ||
| * so a multi-target run would bake that one binary into outputs for other | ||
| * platforms/arches — silently producing broken artifacts. We: | ||
| * | ||
| * 1. Reject if the requested targets span more than one distinct | ||
| * `platform`+`arch` — one binary can't be several mutually-exclusive things | ||
| * at once (incl. linux vs alpine vs linuxstatic, which we can't tell apart | ||
| * from the binary but the user clearly can't have meant simultaneously). | ||
| * 2. Sniff the binary and reject a format/arch mismatch against that single | ||
| * target (e.g. a macOS binary for a `linux` target, or x64 for `arm64`). | ||
| * The glibc/musl/static flavor isn't in the header, so that sub-distinction | ||
| * stays the user's responsibility. | ||
| * 3. Verify the binary's major matches the requested `nodeRange` | ||
| * (`assertSingleTargetMajor` only compares the targets to each other). | ||
| */ | ||
| async function assertCustomBaseNodeTarget( | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [Nit] · Types
|
||
| targets: (NodeTarget & Partial<Target>)[], | ||
| opts: GetNodejsExecutableOptions, | ||
| ): Promise<void> { | ||
| const customNode = resolveCustomBaseNode(opts); | ||
| if (!customNode && !opts.useLocalNode) return; | ||
| const binPath = customNode ?? process.execPath; | ||
|
|
||
| // 1. A single binary maps to exactly one platform+arch. Key on the raw target | ||
| // suffix strings so linux / alpine / linuxstatic stay distinct (they collapse | ||
| // under getNodeOs, but they're mutually exclusive runtimes). | ||
| const combos = new Map<string, { platform: string; arch: string }>(); | ||
| for (const t of targets) { | ||
| const platform = String(t.platform); | ||
| const arch = String(t.arch); | ||
| combos.set(`${platform}-${arch}`, { platform, arch }); | ||
| } | ||
| if (combos.size > 1) { | ||
| throw wasReported( | ||
| `A custom base Node binary applies to a single platform/arch, but the ` + | ||
| `requested targets span ${combos.size}: ${[...combos.keys()].join(', ')}. ` + | ||
| `One binary can't be all of them — run pkg once per target with a ` + | ||
| `matching binary.`, | ||
| ); | ||
| } | ||
| const { platform, arch } = [...combos.values()][0]; | ||
|
|
||
| // 2. Format / arch match against that single target. | ||
| const sniff = await sniffBinaryTarget(binPath); | ||
| if (sniff.format) { | ||
| const expected = formatForPlatform(platform); | ||
| if (sniff.format !== expected) { | ||
| throw wasReported( | ||
| `Custom base Node binary is ${FORMAT_LABEL[sniff.format]}, but target ` + | ||
| `"${platform}" needs ${FORMAT_LABEL[expected]}.`, | ||
| ); | ||
| } | ||
| if (sniff.arch && sniff.arch !== arch) { | ||
| throw wasReported( | ||
| `Custom base Node binary is ${sniff.arch}, but target arch is "${arch}". ` + | ||
| `The binary must match the target's architecture.`, | ||
| ); | ||
| } | ||
| } | ||
|
|
||
| // 3. Major version match. | ||
| const targetMajor = parseInt(targets[0].nodeRange.replace('node', ''), 10); | ||
| if (Number.isNaN(targetMajor)) return; // 'latest' / unparseable: nothing to compare | ||
| const version = | ||
| binPath === process.execPath | ||
| ? process.version | ||
| : (await execFileAsync(binPath, ['--version'])).stdout.trim(); | ||
| const binMajor = parseInt(version.replace(/^v/, ''), 10); | ||
| if (binMajor !== targetMajor) { | ||
| throw wasReported( | ||
| `Custom base Node binary is ${version} (major ${binMajor}), but target ` + | ||
| `"${targets[0].nodeRange}" requests Node ${targetMajor}. The binary's ` + | ||
| `major version must match the target.`, | ||
| ); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Resolve the concrete Node.js version (e.g. `v22.22.2`) pkg will use | ||
| * for `target` — mirrors the version selection done inside | ||
|
|
@@ -341,10 +537,11 @@ async function resolveTargetNodeVersion( | |
| opts: GetNodejsExecutableOptions, | ||
| ): Promise<NodeVersion> { | ||
| if (opts.useLocalNode) return process.version as NodeVersion; | ||
| if (opts.nodePath) { | ||
| const customNode = resolveCustomBaseNode(opts); | ||
| if (customNode) { | ||
| // A user-supplied binary can be any version — don't assume it | ||
| // matches the host. Ask it directly. | ||
| const { stdout } = await execFileAsync(opts.nodePath, ['--version']); | ||
| const { stdout } = await execFileAsync(customNode, ['--version']); | ||
| return stdout.trim() as NodeVersion; | ||
| } | ||
| const os = getNodeOs(target.platform); | ||
|
|
@@ -357,15 +554,16 @@ async function getNodejsExecutable( | |
| target: NodeTarget, | ||
| opts: GetNodejsExecutableOptions, | ||
| ): Promise<string> { | ||
| if (opts.nodePath) { | ||
| // check if the nodePath exists | ||
| if (!(await exists(opts.nodePath))) { | ||
| const customNode = resolveCustomBaseNode(opts); | ||
| if (customNode) { | ||
| // check if the custom base binary exists | ||
| if (!(await exists(customNode))) { | ||
| throw new Error( | ||
| `Priovided node executable path "${opts.nodePath}" does not exist`, | ||
| `Provided node executable path "${customNode}" does not exist`, | ||
| ); | ||
| } | ||
|
|
||
| return opts.nodePath; | ||
| return customNode; | ||
| } | ||
|
|
||
| if (opts.useLocalNode) { | ||
|
|
@@ -765,6 +963,7 @@ export async function seaEnhanced( | |
| } | ||
|
|
||
| assertSingleTargetMajor(opts.targets); | ||
| await assertCustomBaseNodeTarget(opts.targets, opts); | ||
|
|
||
| const minTargetMajor = resolveMinTargetMajor(opts.targets); | ||
| if (minTargetMajor < 22) { | ||
|
|
@@ -883,6 +1082,7 @@ export async function seaEnhanced( | |
| export default async function sea(entryPoint: string, opts: SeaOptions) { | ||
| assertHostSeaNodeVersion(); | ||
| assertSingleTargetMajor(opts.targets); | ||
| await assertCustomBaseNodeTarget(opts.targets, opts); | ||
|
|
||
| entryPoint = resolve(process.cwd(), entryPoint); | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| 'use strict'; | ||
|
|
||
| // Packaged into a SEA on top of a custom base Node binary (the host's own node, | ||
| // supplied via --sea-node-path or PKG_NODE_PATH). Output is asserted by main.js; | ||
| // process.pkg confirms the enhanced SEA bootstrap is active. | ||
| console.log('custom-node SEA OK:' + (process.pkg != null)); |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[Nit] · Consistency
PKG_NODE_PATHis passed throughresolve()(line 345) butopts.nodePath(the--sea-node-pathflag) is returned verbatim — so a relative--sea-node-pathis left un-normalized. Harmless today (exec/exists resolve against cwd) but inconsistent.