Skip to content

fix(gateway): detect stopped processes and release stale locks on --replace - #2189

Closed
ygd58 wants to merge 1 commit into
NousResearch:mainfrom
ygd58:fix/gateway-replace-stopped-process
Closed

fix(gateway): detect stopped processes and release stale locks on --replace#2189
ygd58 wants to merge 1 commit into
NousResearch:mainfrom
ygd58:fix/gateway-replace-stopped-process

Conversation

@ygd58

@ygd58 ygd58 commented Mar 20, 2026

Copy link
Copy Markdown
Contributor

Fixes #2178

Root Cause

Two bugs when hermes gateway run --replace encounters a STOPPED (Ctrl+Z) process:

  1. acquire_scoped_lock() treated stopped processes as alive — os.kill(pid, 0) succeeds for stopped processes, so stale lock files were never cleaned up
  2. start_gateway() with --replace removed the PID file but not the scoped lock files in ~/.local/state/hermes/gateway-locks/

Fix (Option C: both fixes)

gateway/status.py — Option B:

  • After os.kill(pid, 0) succeeds, check /proc/<pid>/status for process state
  • If state is T (stopped) or t (tracing stop), treat as stale and clean up the lock

gateway/status.py — Option A:

  • Added release_all_scoped_locks() function that removes all lock files in the lock directory

gateway/run.py:

  • After remove_pid_file() in --replace logic, call release_all_scoped_locks() to clean up stale locks from the old gateway

@teknium1

Copy link
Copy Markdown
Contributor

Merged via PR #2406. Cherry-picked with authorship preserved. Thanks!

@teknium1 teknium1 closed this Mar 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] hermes gateway run --replace fails when old gateway process is STOPPED (Ctrl+Z)

2 participants