Sanitize playground output, unblock event loop, scope routing result per request - #30
Open
Sajjad-Fazeli wants to merge 1 commit into
Open
Conversation
…uting result per request Three runtime/security fixes in the serving adapters: XSS: the playground rendered model output via marked.parse() straight into innerHTML with no sanitizer, and interpolated judge verdicts, model display names, and status strings into innerHTML unescaped. A model response (or a prompt-injected one) containing markup could execute script in the browser. Markdown is now sanitized with DOMPurify and all other untrusted interpolations are HTML-escaped; both paths fall back to textContent if a CDN library is unavailable. Event-loop blocking: the SSE chat endpoint and the router-only /v1/route endpoint called router.route() — a synchronous encoder forward pass — directly on the event loop, stalling all concurrent requests for its duration. Both now dispatch through asyncio.to_thread, matching the custom routing strategy. Cross-request races: the strategy stored the routing decision on a single shared attribute that completions.py and the proxy middleware read after awaiting the upstream call, so concurrent requests could read each other's selected model and stamp the wrong model onto a response. The result is now held in a request-scoped contextvar slot installed via begin_request(); last_result falls back to the shared attribute when no slot is set, preserving existing behavior.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three runtime/security fixes in the serving adapters (full-mode app, router-only sidecar, and LiteLLM proxy middleware).
1. Playground XSS
The playground rendered streamed model output with
marked.parse()directly intoinnerHTMLwith no sanitizer, and interpolated judge verdicts, model display names, confidence labels, and status strings intoinnerHTMLunescaped. A model response — or a prompt-injected one — containing HTML such as<img src=x onerror=...>would execute script in the browser.esc()helpertextContentif a CDN library fails to load, so raw model output is never injected2. Event-loop blocking
The SSE
/api/chatendpoint and the router-only/v1/routeendpoint calledrouter.route()— a synchronous encoder forward pass — directly on the event loop, stalling every concurrent request for its full duration. Both now dispatch throughasyncio.to_thread, matching what the custom routing strategy already does inasync_get_available_deployment.3. Cross-request routing-result race
The strategy stored each routing decision on a single shared
_last_resultattribute.completions.pyand the proxy response middleware read it back after awaiting the upstream completion to attach routing metadata / rewrite the responsemodelfield — so under concurrency, request A could read request B's decision and report the wrong selected model.contextvarslot installed viastrategy.begin_request()(same pattern the per-request tolerance override already uses)last_resultfalls back to the shared attribute when no slot is installed, preserving existing single-request behavior and the library APIChanges
static/playground.js:esc()+renderMarkdown()helpers; escape/sanitize every model-, judge-, and config-derived value written toinnerHTMLstatic/index.html: load DOMPurify alongside markedadapters/litellm/chat.py,adapters/http/route.py: route viaasyncio.to_threadadapters/litellm/strategy.py: request-scoped result slot (begin_request(),_record_result(), contextvar-awarelast_result)adapters/litellm/completions.py,adapters/litellm/proxy.py: callbegin_request()before dispatchlast_resultstill works withoutbegin_request()Tests
python -m pytest tests/adapters/test_litellm.py -q— 15 passedpython -m pytest -q— 227 passed, 30 skipped; the 6 failures are pre-existing onv3(4 are the httpxASGITransportincompatibility addressed by Harden playground streaming for reasoning chunks #26, plus environment-specifictest_gpu/test_trunk)python -m ruff check src/model_router_toolkit/adapters/ tests/adapters/test_litellm.py— cleannode --check src/model_router_toolkit/adapters/litellm/static/playground.js— clean