Skip to content

fix(runtime): reclaim orca-runtime.json when it stops describing this runtime - #10840

Merged
brennanb2025 merged 3 commits into
mainfrom
brennanb2025/sta1513-macos-stale-bootstrap
Jul 28, 2026
Merged

fix(runtime): reclaim orca-runtime.json when it stops describing this runtime#10840
brennanb2025 merged 3 commits into
mainfrom
brennanb2025/sta1513-macos-stale-bootstrap

Conversation

@brennanb2025

Copy link
Copy Markdown
Contributor

Addresses #7848 (STA-1513) — the macOS manifestation. Companion to #10463, which fixes the Linux one. Different mechanism, no file overlap; both are needed.

Problem

On macOS a second launch does not lose the single-instance lock — it can win it. Chromium's lock depends on the profile's SingletonSocket / SingletonCookie, and the socket those point at lives under $TMPDIR (/var/folders/.../T). When those entries go missing, a second launch acquires the lock while the owner is alive, completes startup, and republishes orca-runtime.json with its own pid. Once it exits, the CLI is pointed at a dead pid while the original app keeps serving — stale_bootstrap / app.running: false / runtime_unavailable against a healthy app.

This is not the lock-loser path #10463 guards: that PR's guard does not apply here, because the second instance genuinely holds the lock.

Reproduction

Probe matrix, owner alive throughout (S1 is the control):

# Mutation while the owner holds the lock Second launch Record clobbered
S1 none (control) lock=false, quits pre-ready no
S2 rm -rf the $TMPDIR/scoped_dir* holding SingletonSocket lock=true, runs whenReady YES
S3 rewrite SingletonLockOtherHost.local-<pid> lock=false, quits pre-ready no
S4 delete the SingletonSocket file, keep its dir lock=true YES
S5 delete the SingletonSocket symlink lock=true YES
S6 delete the SingletonCookie symlink lock=true YES

Confirmed with two real Orca main processes on one isolated profile, not just the probe:

A pid=68716 -> orca status: "ready", reachable true
rm -rf <$TMPDIR scoped dir holding SingletonSocket>
B pid=70114 -> acquires the lock, rewrites orca-runtime.json to pid 70114
kill -9 70114
=> A still alive and serving; record still names 70114
=> orca status --json : app.running false, runtime.state "stale_bootstrap"
=> orca terminal list : runtime_unavailable

Byte-for-byte the reported output, including at v1.4.121 (the reported version).

Why the files go missing (hypothesis, not observed): com.apple.bsd.dirhelper runs daily at 03:35 with CLEAN_FILES_OLDER_THAN_DAYS=3 over the per-user /var/folders/*/*/T tree, by access time. A live Orca's SingletonSocket is only accessed when a second launch tries to notify it, so on a Mac left running 3+ days it is a purge candidate. Orca never touches Singleton* itself (grepped) — the deletion always comes from outside the app. I simulated the deletion; I did not watch macOS perform it.

Fix

Rather than trying to enumerate every way the lock can be defeated, the runtime owner now watches its own discovery record and reclaims it when it stops describing a live runtime. This is mechanism-agnostic: it repairs the record regardless of what clobbered it.

Ping-pong safety is the critical property: reclaim happens only when the recorded pid is not running, so two live runtimes sharing a profile leave each other alone. isPidRunning treats only ESRCH as dead (EPERM means a foreign owner holds the pid), matching the existing socket-sweep rule. Live-verified: 25s of coexistence with no ping-pong, then reclaim ~2s after kill -9.

Shutdown is covered twice: stop() clears activeTransports before stopping the watch, and republish no-ops when there are no active transports, so a poll landing mid-teardown cannot resurrect a record. The timer is unref'd so discovery bookkeeping never keeps the process alive.

Ruled out

Hostname change in SingletonLock (S3), websocket-only / null-authToken (the status.ts Path 1 branch), the 104-byte sun_path limit, and corrupt records. A genuine lock loser on macOS still dies pre-ready (S1), which re-confirms the platform fact established in #10463.

Known gap, deliberately not fixed here

src/main/index.ts:2456 still swallows a runtimeRpc.start() rejection on the desktop path. The new watch does not cover it, because the watch arms only after a successful publish. An app that runs with no discovery record at all is a separate robustness bug and deserves its own change.

Test plan

  • New runtime-metadata-ownership-watch unit tests: leaves its own record alone, reclaims missing/dead/unreadable records, reclaims a foreign runtimeId stamped on this pid, yields to another live runtime so two instances cannot ping-pong, polls without an explicit check, stops on stop(), and keeps polling after a republish failure.
  • runtime-rpc tests for republish-after-clobber and delete-underneath.
  • vitest src/main/runtime/2667 passed, 0 failed. typecheck + oxlint clean.

Full investigation report: docs/bug-reproductions/sta-1513-macos-stale-bootstrap.md.

… runtime

On macOS the Chromium single-instance lock is silently defeated whenever
`SingletonSocket`/`SingletonCookie` go missing from the profile — and the
socket they point at lives under `$TMPDIR` (`/var/folders/.../T`), which
macOS purges after 3 days (`com.apple.bsd.dirhelper`,
CLEAN_FILES_OLDER_THAN_DAYS=3). A launch that slips past the lock runs a
full startup, republishes `orca-runtime.json` with its own pid, and leaves
the CLI on a dead pid once it exits: `orca status` reports
`stale_bootstrap` and every terminal command fails `runtime_unavailable`
while the original app keeps serving.

The owner now watches its own discovery record and republishes once no live
runtime is described. Reclaiming only a dead pid is deliberate: two live
runtimes sharing a profile would otherwise fight over the file.

Reproduced on macOS with two real Orca main processes on one profile: the
second instance took the lock and clobbered the record, and killing it left
`stale_bootstrap` against the still-healthy first instance. With this
change the owner reclaimed the record in ~2s and the CLI returned to
`ready`.

Refs #7848
@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 355ab829-cc32-4e97-8c80-124b1fdeb462

📥 Commits

Reviewing files that changed from the base of the PR and between 845d82a and 0ce364a.

📒 Files selected for processing (1)
  • src/main/runtime/runtime-rpc.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/main/runtime/runtime-rpc.test.ts

📝 Walkthrough

Walkthrough

Adds runtime metadata ownership detection and recovery. The watcher evaluates metadata identity and PID liveness, polls for changes, handles unreadable records, retries failed republishing, and exposes check/stop controls. The RPC server starts the watcher after publishing metadata and stops it during shutdown. Tests cover reclaim decisions, polling behavior, republishing failures, live sibling runtimes, and stopped servers.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description is narrative, but it misses the template’s Summary, Screenshots, Testing checklist, AI Review Report, Security Audit, and Notes sections. Reformat it to the repository template and add each required section, including testing status, AI review, security audit, screenshots/no visual change, and notes.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly states the core fix: reclaiming orca-runtime.json when it no longer matches the current runtime.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 38f30a93-d727-40fe-8dcf-244cbca2e979

📥 Commits

Reviewing files that changed from the base of the PR and between 3200f72 and 753c74d.

📒 Files selected for processing (4)
  • src/main/runtime/runtime-metadata-ownership-watch.test.ts
  • src/main/runtime/runtime-metadata-ownership-watch.ts
  • src/main/runtime/runtime-rpc.test.ts
  • src/main/runtime/runtime-rpc.ts

Comment thread src/main/runtime/runtime-rpc.test.ts
The republish guard alone kept the shutdown test green, so the watch teardown was unasserted. Also drop the doc claim of startup/activation callers that do not exist.
Windows never assigns pid 1, so the hardcoded sibling read as dead there and
the watch would reclaim the record. Own a synthetic pid instead and let
process.pid play the live sibling.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant