v3.0 Planning: Universal Proxy Direction

[askalf]

v3.0 Vision

With v2.x stable (OpenAI compat, proxy auth, security hardening), this thread is for discussing what a v3.0 could look like.

Ideas on the table

• CLI backend improvements — deeper integration with Claude Code binary for routing flexibility
• Presence heartbeat — session-aware routing for better rate limit behavior
• Beta flag forwarding — pass through client beta flags to upstream
• Multi-provider support — proxy to other LLM providers beyond Anthropic
• Plugin system — middleware hooks for request/response transformation
• Config file — YAML/JSON config instead of CLI flags only

Context

Some of these ideas came from PR #2 (security audit by @GodsBoy) which included CLI backend, presence heartbeat, and beta flag changes. The security improvements were cherry-picked into v2.1.0, and the feature ideas are worth discussing here for a potential v3.0.

Contributing

Drop your ideas, use cases, or concerns below. The project's value is that it's small enough to audit (~1,500 lines), so any v3.0 work needs to maintain that philosophy.

See CONTRIBUTING.md for contribution guidelines.

[GodsBoy]

One quick correction first: PR #2 was opened by @GodsBoy, not @therealbryantt as the discussion body currently says.

On the substance: if v3 is meant to support Hermes and OpenClaw well, I think the goal should be an explicit compatibility contract, not just a feature list.

For Hermes/OpenClaw, the default proxy mode should behave like a thin Anthropic-compatible transport layer:

• preserve request/response structure as much as possible
• preserve SSE streaming semantics
• preserve Anthropic error envelopes unless there is a strong security reason to normalize them
• forward client-requested Anthropic headers/flags unless there is a hard incompatibility
• inject only the minimum OAuth-specific requirements needed for subscription auth

That matters because both Hermes and OpenClaw are sensitive to protocol details, not just whether one demo request succeeds.

A few concrete implications:

• anthropic-beta should be passed through exactly, with only required OAuth betas added
• no hardcoded Claude-Code-specific beta bundle in the universal/default mode
• Claude-specific features like CLI backend routing, presence heartbeat, or Claude-Code behavior matching should be opt-in, not default coupling
• preserve Anthropic-native behavior first; layer special modes on top of that

For Hermes specifically, live integration testing showed that compatibility depends on more than unit tests:

• Hermes resolves Anthropic base_url from ~/.hermes/config.yaml, not from ANTHROPIC_BASE_URL
• Hermes sends its own x-api-key, and the proxy has to handle that cleanly when swapping in upstream OAuth bearer auth
• streaming compatibility only really counts if /v1/messages SSE framing survives a real Hermes runtime

So my vote is:

1. make proxy the universal/default mode
2. keep Claude-optimized behavior behind an explicit mode/profile
3. define v3 in terms of protocol compatibility guarantees for Hermes/OpenClaw, not just internal implementation ideas

[GodsBoy]

A second point: I think v3 should have an explicit compatibility test matrix and treat that as a release gate.

If Dario wants to claim it is Hermes/OpenClaw compatible, that should mean live end-to-end verification, not just unit coverage.

At minimum I would want this matrix:

Anthropic Messages compatibility

• Hermes -> Dario -> Anthropic: /v1/messages non-streaming
• Hermes -> Dario -> Anthropic: /v1/messages streaming SSE
• OpenClaw -> Dario -> Anthropic: standard request path
• OpenClaw -> Dario -> Anthropic: header/beta-sensitive request path

OpenAI compatibility

• OpenAI-compatible client -> Dario /v1/chat/completions non-streaming
• OpenAI-compatible client -> Dario /v1/chat/completions streaming

What should be asserted

• request reaches the proxy and upstream successfully
• anthropic-version is preserved
• anthropic-beta is merged/preserved correctly
• downstream auth headers do not break upstream OAuth forwarding
• Anthropic rate-limit / request-id headers are still visible to the client
• SSE events/chunks arrive in the expected order and shape
• error responses remain useful enough for client runtimes to branch on

For Hermes specifically, the live integration runbook I used showed a few things that are easy to miss unless they are tested explicitly:

• Hermes reads Anthropic base_url from ~/.hermes/config.yaml, not from .env
• SSE needs to be validated over a real HTTP stack
• compatibility can be broken by config resolution or header behavior even when unit tests pass

I also think this test matrix helps keep scope under control.

If the project value is that it stays small enough to audit, I would strongly prefer:

1. universal Anthropic/OpenAI proxy compatibility first
2. Claude-specific extras isolated behind explicit modes
3. multi-provider/plugin work only after the compatibility surface is stable

That ordering seems like the best way to make “universal proxy” real without turning v3 into a grab bag.

[askalf]

Fixed the attribution — sorry about that @GodsBoy.

On the architecture: I agree with the compatibility contract framing, but I think the default should be inverted from what you're proposing.

dario's core value is that it's the only proxy that bills correctly and keeps Opus/Sonnet available at high utilization. That requires injecting device identity, billing classification tags, adaptive thinking, and service_tier — without those, you get the same 429s and billing misrouting that every other proxy has. We just spent a full day proving this via MITM capture and A/B testing (see Discussion #1).

So the default should stay Claude-optimized. But a --mode=passthrough (or --thin) that does exactly what you describe — minimal injection, just OAuth swap — makes complete sense as an explicit mode. That's the right escape hatch for tools like Hermes/OpenClaw that need exact protocol fidelity.

On the test matrix: fully agree. We just built a 12-test E2E suite today (Haiku/Sonnet/Opus, streaming/non-streaming, OpenAI compat, tool use, rate limit headers — all green). Adding Hermes and OpenClaw as live integration targets for v3 is the right move. Your specific assertions (header preservation, SSE ordering, error envelope fidelity) are exactly what we'd test.

Proposed v3 mode split:

• default — Claude-optimized (billing tag, device identity, thinking, service_tier, context_management, beta flags)
• --passthrough — thin transport layer (OAuth swap only, forward everything else as-is)
• --cli — existing CLI backend

That keeps the product's key differentiator as the default while giving protocol-sensitive tools a clean path.

[askalf]

Update: --passthrough shipped in v2.8.0 (live on npm).

dario proxy --passthrough    # OAuth swap only, zero injection

This is the thin transport layer mode discussed above — no billing tag, no thinking, no service_tier, no device identity, no extra betas. Just swaps your API key for the OAuth bearer token and forwards everything else as-is.

Ready for Hermes/OpenClaw integration testing. If anyone wants to run it against their setup and report back, that would help shape the v3 compatibility contract.

Also in v2.8.0:

• CLI streaming — --cli now returns SSE when client sends stream: true
• E2E test suite — npm run e2e (12 tests, basis for the compatibility matrix discussed above)
• Enriched 429 errors — rate limit responses now include utilization %, window, and reset time

[askalf]

Comparison test: Default vs Passthrough (post-7d reset)

Ran the same test suite through both modes back-to-back. 7-day window just reset, so this is a clean baseline.

Default mode (Claude-optimized)

Haiku:  200 | thinking=none | tier=-
Sonnet: 200 | thinking=none | tier=-
Opus:   200 | thinking=none | tier=-
Sonnet stream: 200 | 12 events | valid
Opus stream:   200 | 11 events | valid
OpenAI compat: 200

Rate limits (Sonnet):
  status: allowed
  5h-utilization: 0.33
  7d-utilization: 0.02
  7d_sonnet-utilization: 0.05
  representative-claim: five_hour
  fallback-percentage: 0.5
  overage-utilization: 0.0

Passthrough mode (--passthrough)

Haiku:  200 | thinking=none | tier=-
Sonnet: 200 | thinking=none | tier=-
Opus:   200 | thinking=none | tier=-
Sonnet stream: 200 | 12 events | valid
Opus stream:   200 | 11 events | valid
OpenAI compat: 200

Rate limits (Sonnet):
  status: allowed
  5h-utilization: 0.34
  7d-utilization: 0.02
  7d_sonnet-utilization: 0.05
  representative-claim: five_hour
  fallback-percentage: 0.5
  overage-utilization: 0.0

Observations

1. Both modes work identically at low utilization — at 2% 7d / 34% 5h, there's no functional difference
2. Passthrough confirms zero injection — no thinking blocks in responses, no billing tag, no device identity
3. Per-model headers returned in both modes — the server reports 7d_sonnet utilization regardless of mode, but routing decisions differ at high utilization
4. Streaming works in both — SSE event structure is identical
5. The real difference shows at >75% 7d utilization — that's when default mode's billing tag activates per-model routing (Sonnet at 5% passes) while passthrough would evaluate against overall 7d (100% = 429)

@GodsBoy — passthrough is ready for Hermes testing. At low utilization it's a clean transport layer. The trade-off is at high utilization where default mode's priority routing keeps Opus/Sonnet alive.

[GodsBoy]

This is a good outcome.

I am fine with the default staying Claude-optimized as long as --passthrough is treated as the canonical compatibility mode for protocol-sensitive clients.

That makes the next question straightforward: not "which architecture is right," but "does --passthrough actually satisfy Hermes/OpenClaw in practice?"

For me, the acceptance bar would be:

Hermes

• provider: anthropic with base_url pointed at dario works without extra glue
• /v1/messages non-streaming works
• /v1/messages streaming works with correct SSE ordering/framing
• Hermes own auth/header behavior does not conflict with upstream OAuth swap
• no hidden dependency on Claude-specific injected fields

OpenClaw

• standard Anthropic request flow works in --passthrough
• beta/header-sensitive paths work in --passthrough
• tool-use / structured-content paths pass through cleanly
• rate-limit/request-id headers remain visible enough for client behavior/debugging

Compatibility claim

I would only call Hermes/OpenClaw "supported" once those are validated live, not just inferred from unit/E2E coverage against synthetic clients.

So I think the right next move is:

1. treat --passthrough as the compatibility target
2. run live Hermes and OpenClaw validation against it
3. turn the results into an explicit compatibility contract in docs/tests

I am happy to help by running Hermes/OpenClaw against v2.8.0 --passthrough and reporting exact failures or confirmations.

[askalf]

Agree 100% with the validation-first approach. No point claiming compatibility without live proof.

v2.8.1 just shipped with a code cleanup pass (1,618 → 1,505 lines, −7%). Also caught that Haiku doesn't support output_config.effort — same pattern as thinking, now skipped for Haiku in both default and passthrough modes. 12/12 E2E tests passing.

On your offer to run Hermes/OpenClaw against --passthrough — yes please. That's exactly the right next step. Here's what would be most useful:

1. Hermes: point base_url at dario proxy --passthrough, run your normal workflow. Note any failures, unexpected headers, or behavioral differences vs direct API.
2. OpenClaw: same setup. Especially interested in tool-use paths and whether the minimal beta set (oauth-2025-04-20 only) causes any issues.
3. Report format: exact error messages, request/response snippets if anything breaks. Even "it just worked" is valuable data.

Once we have live results, I'll add the compatibility matrix to the README and codify it in the E2E suite as a release gate — exactly as you proposed.

npm i -g @askalf/dario@2.8.1 to get the latest.

[GodsBoy]

OK will do and come back once done properly.

[askalf]

Compatibility validation suite + first results

Pushed test/compat.mjs — a 10-test suite specifically for --passthrough protocol validation. Run it with:

dario proxy --passthrough &
npm run compat

Test matrix

#	Test	What it validates
1	Anthropic non-stream	Basic /v1/messages request/response
2	Anthropic stream	SSE event sequence (start → delta → stop)
3	SSE framing	Every data: line has preceding event: line
4	No thinking injection	Passthrough doesn't inject thinking blocks
5	Client betas preserved	Client-sent beta headers forwarded correctly
6	Tool use	tool_choice: any returns tool_use block
7	Tool use streaming	Streaming tool_use with input_json_delta
8	OpenAI non-stream	/v1/chat/completions translation
9	OpenAI stream	OpenAI SSE format with [DONE]
10	Header visibility	request-id and ratelimit headers forwarded

First results (ran at 44% 5h utilization)

Every request 429'd in passthrough and fell back to CLI — which is exactly the point. Without the billing tag, standard rate limits apply and you get throttled at high utilization.

✅ #1-5  Anthropic non-stream, stream, SSE framing, no injection, client betas — all PASS (via CLI fallback)
❌ #6-7  Tool use — FAIL (CLI backend doesn't support tools)
✅ #8-9  OpenAI compat — PASS (via CLI fallback)
⚠️ #10   Header visibility — WARN (CLI responses don't have Anthropic rate limit headers)

What this tells us

1. Passthrough protocol is clean — no injection verified, client betas forwarded, SSE framing correct
2. Rate limiting is real — at 44% 5h utilization, passthrough gets 429'd while default mode (with billing tag + service_tier: auto) gets 200s on the same account
3. Tool use needs direct API — CLI fallback doesn't support tools, so tool-heavy clients (OpenClaw) need to run when rate limits allow, or use default mode

When you run Hermes/OpenClaw against it, the cleanest test would be during low utilization (after a 5h window reset) so requests hit the API directly rather than CLI fallback. That'll give you the real protocol compatibility data.

npm i -g @askalf/dario@2.8.1 has the suite.