feat: TCP keepalive, reverse-channel probe, list_models cache, .pth atomic write, launch exemption - #4845
Merged
qinxuye merged 3 commits intoApr 26, 2026
Conversation
…tomic write, launch exemption - Add xoscar TCP keepalive monkey-patch (SO_KEEPALIVE + KEEPIDLE=60s) to prevent idle connections from being silently dropped by firewalls - Add supervisor->worker reverse-channel ping probe in _check_dead_nodes to detect dead reverse channels even when heartbeat is healthy - Add list_models cache degradation: return cached results when a worker is temporarily unreachable instead of returning empty dict - Reduce XINFERENCE_GET_MODEL_RPC_TIMEOUT from 120s to 30s to limit actor lock contention during worker communication failures - Fix .pth file concurrent write race in virtualenv setup by using atomic write (os.replace) and skipping writes when content unchanged - Exempt workers from reverse-channel dead detection while launching models, as long downloads can starve the actor event loop - Limit huggingface_hub download concurrency to 2 (XINFERENCE_MODEL_DOWNLOAD_WORKERS) to reduce GIL contention during model downloads Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
qinxuye
reviewed
Apr 24, 2026
xoscar v0.9.6 natively implements TCP keepalive in SocketClient.connect()
with identical parameters (KEEPIDLE=60s, KEEPINTVL=10s, KEEPCNT=3) and
additionally covers Windows. The xinference monkey-patch is now redundant.
- Upgrade xoscar minimum version from 0.9.3 to 0.9.6
- Delete xinference/core/xoscar_keepalive.py
- Remove keepalive import from deploy/{worker,supervisor,local}.py
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_check_dead_nodesto detect dead reverse channels even when worker→supervisor heartbeat is healthylist_modelscache degradation: return cached results when a worker is temporarily unreachable instead of returning empty dict (prevents UI from showing no models)XINFERENCE_GET_MODEL_RPC_TIMEOUTfrom 120s to 30s to limit actor lock contention during worker communication failures.pthfile concurrent write race in virtualenv setup by using atomic write (os.replace) and skipping writes when content is unchangedhuggingface_hubdownload concurrency to 2 (XINFERENCE_MODEL_DOWNLOAD_WORKERS) to reduce GIL contention during model downloadsChanged Files
setup.cfgxinference/core/supervisor.py_workers_launchingtrackingxinference/core/worker.pyping()method,.pthatomic write, download concurrency limitxinference/constants.pyXINFERENCE_MODEL_DOWNLOAD_WORKERSTest plan
ss -tnop | grep <supervisor-port>showstimer:(keepalive,...)