Research node-swift as a replacement for the shell-out npm path

[leogdion]

Moved from brightdigit/brightdigit.com#51, which was a bare link with no body.

https://github.com/kabiroberai/node-swift

Why it lands here

node-swift provides Swift↔Node interop (write Node native modules in Swift / call into Node from Swift). NPMPublishPlugin is the package that actually talks to npm, so any evaluation belongs against this codebase rather than the site repo.

What to actually evaluate

• Whether NPMPublishPlugin's current npm invocation is a shell-out (it depends on ShellOut directly) and whether node-swift could replace that with in-process interop
• Platform cost: node-swift requires a Node toolchain at build/run time — check what that does to the Linux/Windows/Android/wasm legs. brightdigit.com#78 (now closed) recorded ShellOut as one of the two blockers keeping this plugin off the cross-platform matrix, so this is the relevant tradeoff.
• Whether the benefit justifies the dependency at all, versus keeping the shell-out

Status

Unscheduled research spike, not committed work.

[leogdion]

Cross-reference: the consumer-side tracker is brightdigit/brightdigit.com#51, reopened and kept in that repo's Phase 8 milestone. Decisions land here; that issue tracks the outcome for the site build.

[leogdion]

Research findings

Investigated against main (904d35e) and node-swift 979eca4 (2026-06-22, v1.5.2).

Correction to the premise: the ShellOut dependency is already gone

The issue says the plugin "depends on ShellOut directly." That's stale. Package.swift now depends on swift-subprocess 0.4.x, and NPMInvocation runs npm via Subprocess.run(executable, arguments:workingDirectory:) — no shell, no ShellOut. NPMInvocation.swift says so explicitly:

This replaces the previous ShellOutCommand abstraction. […] The executable and its arguments are passed directly to the process, so no shell is involved: argument values cannot be reinterpreted as shell syntax, and paths containing spaces need no escaping.

So the injection/quoting class of problem that usually motivates "stop shelling out" is already solved. What remains is a plain posix_spawn, and the dependency is already platform-conditioned to [.macOS, .linux, .windows, .android].

This also partly retires the brightdigit.com#78 framing: the blocker recorded there was ShellOut, and ShellOut is no longer in the graph. Whatever remains of the cross-platform question is about swift-subprocess's platform matrix, not ShellOut's.

node-swift runs in the wrong direction for this use case

This is the blocking finding, and it's architectural rather than a matter of polish.

node-swift builds Swift native modules that Node loads — JS is the host process, Swift is the guest. Sources/CNodeAPI is a SwiftPM .systemLibrary wrapping a vendored node_api.h header with no library to link against; the Node-API symbols are left deliberately undefined and resolved by the hosting Node process at dlopen time. src/builder.ts confirms this — it links with -undefined dynamic_lookup on Darwin, bare -undefined on Linux, and /DELAYLOAD:node.exe on Windows. The build product is libFoo.dylib/.so/.dll renamed to .node, consumed as require("./build/MyModule.node").

NPMPublishPlugin needs the opposite: a Swift process (Publish's build tool) that drives npm. Nothing in node-swift creates or embeds a Node runtime. The only environment-construction entry point in the whole package is napi_env_jsc_create in NodeJSC, and that binds Node-API to JavaScriptCore, not to Node — no libnode, no node::InitializeOncePerProcess, no platform bootstrap. NodeJSC gives you a JS engine, not npm, and npm cannot run on bare JSC (it needs Node's fs, child_process, module resolution, and native TLS).

Adopting node-swift would therefore mean inverting the process model: Publish would have to become a Node-hosted module, or ship a Node host that loads a Swift .node that calls back into Publish. That's a far larger change than replacing the Subprocess.run call, and it makes the Node toolchain a hard dependency of the whole build rather than a runtime requirement of one step.

Even with the direction solved, there is no npm API to call

The plugin runs exactly two subcommands — ci and run (NPM/Job.swift, plus arbitrary user-supplied Commands via ExpressibleByStringLiteral). Both are CLI-only surface area:

• npm's programmatic API was removed in npm v8.0.0 (npm/cli#3762); require("npm") now throws at module scope. npm's position is that it was never really supported.
• The replacements are per-task libraries — @npmcli/arborist, pacote, libnpmpublish, libnpmexec. npm ci maps roughly onto Arborist's reify, but "roughly" is doing real work there: lockfile semantics, workspaces, lifecycle scripts, and .npmrc/auth resolution are behaviors users get for free from the CLI and would now be ours to reimplement and keep in sync across npm majors.
• npm run is worse. It's a lifecycle-script runner that spawns a shell subprocess by design. Driving it through Node-API would mean Swift → Node → child_process → shell — strictly more process boundaries than today's single spawn, while adding a shell back in that we currently avoid.

So the in-process win doesn't materialize: the two operations we perform either have no stable library equivalent or bottom out in a subprocess anyway.

Platform cost

node-swift's builder.ts hard-fails on anything outside darwin/linux/win32, so Android and wasm are out — the same two legs the cross-platform matrix cares about. The package declares only .macOS(.v10_15), .iOS(.v13). It also pulls in swift-syntax (for its macros), which is a heavy build-time dependency, and requires a Node toolchain plus an npm install-driven node-swift rebuild at build time. Net effect on the matrix: strictly worse than the current conditional Subprocess dependency.

Recommendation

Don't adopt node-swift here, and close this spike as researched. It's a well-built package aimed at a genuinely different problem — enhancing JS tooling with Swift, or pulling npm libraries into an app that is already Node-hosted. Neither describes a Publish build step whose job is to invoke two npm subcommands.

Concretely, the benefit does not justify the dependency:

• the safety motivation is already banked by the swift-subprocess migration;
• the interop direction is inverted and would require restructuring Publish's process model;
• npm exposes no programmatic API for ci/run, and the library replacements would either be reimplemented by us or re-spawn a subprocess regardless;
• Android and wasm regress outright.

If the underlying goal is "get this plugin onto the cross-platform matrix," the productive thread is swift-subprocess's own platform support (already scoped via condition: .when(platforms:)) and what the #if canImport(Subprocess) fallback should do where it's unavailable — today the entire PublishingStep.npm API silently vanishes on unsupported platforms, which is worth an issue of its own.

Worth revisiting only if node-swift grows a Swift-hosts-Node embedding story (a real libnode bootstrap, not the JSC shim), and npm ships a supported programmatic API for ci/run. Both would need to land for the calculus to change.

[leogdion]

Follow-up: alternatives to Subprocess for reaching npm

Correction to my previous comment

I claimed Android regresses under node-swift "the same two legs the cross-platform matrix cares about." The node-swift half stands — src/builder.ts still hard-fails outside darwin/linux/win32. But I understated the baseline: swift-subprocess supports Android with automated CI testing, per its platform table, which I verified against Sources (8 os(Android) guards, plus a dedicated Android import path). Full matrix: macOS, Ubuntu 22.04/24.04, RHEL UBI 9, Debian 12, Amazon Linux 2023, Windows 11, Android, FreeBSD all automated; Static Linux SDK build-only; OpenBSD manual. So the current dependency already covers every platform the plugin declares, and Android is not a gap to close.

The one genuine gap is wasm. Subprocess has exactly one os(WASI) reference — a page-size constant — and WASI is absent from the platform table. That is expected and not fixable by swapping libraries: WASI has no posix_spawn and no process model. No spawning library gets npm running under wasm. If wasm is a real target, the answer is that the npm step doesn't run there, and the design question is what the API does on that platform (see the last section).

Two different questions

"Better alternative for calling npm" splits into two, with different answers:

A. A better way to spawn the npm process. There isn't one, and you're already on the best option.

• Foundation.Process — the thing swift-subprocess is explicitly designed to replace ("a new API that will eventually replace Process as the canonical way to launch a process in Swift"). Blocking API, awkward concurrency story, worse Windows support. A regression.
• posix_spawn/fork+exec directly — hands you exactly the platform-specific mess Subprocess already abstracts (pidfd on Linux, posix_spawn + QoS on Darwin, CreateProcess on Windows). More code, less portability.
• Shelling through sh -c — reintroduces the quoting/injection problem the ShellOut migration removed. Strictly worse.

swift-subprocess is Foundation-project-owned, actively developed, and covers a wider matrix than the plugin needs. The one caveat is that it's pre-1.0 (0.4.x), so source-breaking changes are possible — which the .upToNextMinor(from: "0.4.0") pin in Package.swift already handles correctly.

B. Reaching npm's functionality without spawning the CLI. This is the interesting one, and it's viable for some operations — but not for the two this plugin runs.

npm's own libraries are the real programmatic surface: @npmcli/arborist handles the dependency tree, and npm ci has a documented equivalent — arb.loadVirtual() → remove node_modules → arb.reify(). Arborist handles lockfile v1/v2/v3 and workspaces. libnpmpublish, pacote, and libnpmexec cover publish/fetch/exec.

But these are Node libraries. Calling them requires a JS runtime in-process, which puts us back at the node-swift problem: every Swift↔JS option runs the wrong direction or lacks Node.

• node-swift, Napoli — both build Swift addons loaded by Node. Napoli is explicitly "build Node.js Addons in Swift" for Electron packaging, and is far less maintained (~1 star). Same inverted direction, worse maturity.
• JavaScriptCore / JXKit — embeddable in a Swift process, so the direction is right. But JSC is a bare JS engine: no fs, no child_process, no module resolution, no native TLS. Arborist needs all of them. npm's libraries cannot run on JSC.
• Embedding libnode — the only technically correct route, and nobody ships a Swift package for it. You'd hand-roll node::InitializeOncePerProcess bindings and vendor a ~100MB Node build per platform. Enormously more complexity than one posix_spawn, for two subcommands.

And even granting a working Node host, it doesn't pay off for this plugin: npm ci becomes ours to keep in sync with npm's lockfile/lifecycle/.npmrc semantics across majors, and npm run is a lifecycle-script runner that spawns a shell by design — so interop yields Swift → Node → child_process → shell, more process boundaries than today plus a shell we currently avoid.

Recommendation

Keep swift-subprocess. For invoking npm ci and npm run, spawning the CLI is not a workaround — it is the supported interface. npm removed its programmatic API in v8 precisely so the CLI could stay the contract, and the per-task libraries deliberately don't reconstitute run.

Option B becomes worth revisiting only for a new operation that maps onto a real library — publishing via libnpmpublish is the plausible one — and even then only if a Swift-hosted Node runtime exists. That's a different feature, not a replacement for the current call path.

The actually actionable item

Independent of all the above: #if canImport(Subprocess) currently makes the entire PublishingStep.npm API disappear on platforms without Subprocess, rather than failing with a diagnostic. On wasm — the one platform where this genuinely can't work — a user gets "no such method" instead of "npm steps aren't supported here." Worth splitting into its own issue; it's the real cross-platform defect, and it's fixable today without changing the dependency.

[leogdion]

Outcome: closing as researched

Spike is done. Answering the three questions this issue posed:

Is the npm invocation a shell-out, and could node-swift replace it?
It is no longer a shell-out — the plugin moved from ShellOut to swift-subprocess, and NPMInvocation spawns npm directly with no shell involved. node-swift cannot replace it regardless: it builds Swift native modules that Node loads (JS hosts, Swift is the guest, symbols resolved at dlopen via -undefined dynamic_lookup). The plugin needs the inverse — a Swift process driving npm — and nothing in node-swift embeds a Node runtime. Its only environment entry point binds Node-API to JavaScriptCore, which is a JS engine, not npm.

Platform cost.
Worse than the status quo. node-swift's builder hard-fails outside darwin/linux/win32 and adds a required Node toolchain plus swift-syntax. Meanwhile swift-subprocess already covers macOS, Linux (Ubuntu/RHEL/Debian/Amazon), Windows, Android, and FreeBSD under automated CI — every platform the plugin declares. wasm is the sole genuine gap, and it is unfixable by any spawning library since WASI has no process model.

Does the benefit justify the dependency?
No. The safety motivation was already banked by the swift-subprocess migration, and npm exposes no programmatic API for the two subcommands this plugin runs. npm ci has an Arborist equivalent but would make lockfile/workspace/lifecycle/.npmrc semantics ours to maintain across npm majors; npm run is a lifecycle-script runner that spawns a shell by design, so interop would yield Swift → Node → child_process → shell — more process boundaries than today, plus a shell we currently avoid.

Decision: keep swift-subprocess. For ci and run, spawning the CLI is the supported interface, not a workaround — npm removed its programmatic API in v8 precisely so the CLI stays the contract.

Worth reopening only if node-swift grows a real Swift-hosts-Node embedding story (a libnode bootstrap, not the JSC shim) and npm ships a supported programmatic API for these commands. Both would need to land. A narrower future case — publishing via libnpmpublish — would be a new feature rather than a replacement for this call path, and belongs in its own issue.

One actionable item surfaced that is independent of this decision: #if canImport(Subprocess) currently makes the entire PublishingStep.npm API silently vanish where Subprocess is unavailable, instead of failing with a diagnostic. Filing separately.

cc brightdigit/brightdigit.com#51 — outcome for the site build is "no change; the plugin keeps invoking the npm CLI."