bench(deepseek-v2-lite): retain issue 279 benchmark snapshot

[CAICAIIs]

Summary

Closes #279.

This PR adds a retained DeepSeek-V2-Lite EP2 benchmark snapshot and a reproducible benchmark orchestrator for comparing:

• OpenInfer host-staged
• OpenInfer NCCL
• vLLM TP2
• vLLM TP2+EP2, when the package stack supports it

The snapshot keeps correctness, direct diagnostic batch, HTTP concurrency pressure, OpenInfer trace rows, stock vLLM setup failures, and supplemental vLLM validation as separate claim buckets.

What Changed

• Added scripts/bench_dsv2lite_vllm_matrix.py

  • Runs the HF / host-staged / NCCL correctness gate before performance rows.
  • Runs OpenInfer direct diagnostic batch rows for batch 1/4/8.
  • Uses vllm bench serve as the shared HTTP pressure client.
  • Preserves server-start failures as benchmark evidence instead of dropping failed rows.
  • Supports --plan-only and --summarize-only.

• Added parser/unit coverage in tests/test_bench_dsv2lite_vllm_matrix.py.

• Added the retained benchmark snapshot:

  • docs/benchmarks/deepseek-v2-lite-vllm-tp2-ep2-2026-06.md

• Updated the DeepSeek-V2-Lite status ledger and docs index.

Benchmark Evidence

The retained 2026-06-28 2x RTX 5090 run passed:

• HF dump
• OpenInfer host-staged E2E
• OpenInfer NCCL E2E
• exact comparison: classification=all_token_text_exact
• OpenInfer host-staged/NCCL direct diagnostic batch rows
• OpenInfer host-staged/NCCL HTTP pressure rows
• OpenInfer trace rows

The clean stock vLLM TP2 and TP2+EP2 rows are retained as setup failures on the FlashInfer SM120/CUDA 12.8 path.

A separate FlashInfer #3633-equivalent validation completed the vLLM TP2 and TP2+EP2 HTTP rows under the same client/workload contract. That supplemental run is documented separately from the clean stock-package result.

Claim Boundaries

This PR does not claim:

• vLLM parity
• production DeepSeek-V2-Lite serving readiness
• sparse-dispatch readiness
• multi-node EP support
• full CUDA Graph serving coverage

vllm bench serve --max-concurrency is recorded as HTTP concurrency pressure. It is not treated as internal OpenInfer batch size unless OpenInfer trace rows prove the corresponding engine-side batch behavior.

Validation

• python3 -m unittest tests/test_bench_dsv2lite_vllm_matrix.py
• git diff --check upstream/main..HEAD
• cargo fmt --all --check
• Public docs/scripts privacy scan for local paths, SSH details, hostnames, and credentials

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 750b1f0767

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[xiaguan]

Thanks for preserving this matrix. One thing the retained results seem to highlight is a serving-scheduler/runtime follow-up rather than a benchmark-script issue: OpenInfer reaches active request sets of 4/8, but output throughput stays almost flat while vLLM TP2/TP2+EP2 scales with concurrency.

From the current trace fields, it would be useful to track why active requests are not turning into effective decode throughput. The likely areas to profile are: serial per-request prefill/admission, same-position-only decode subgrouping, per-layer host D2H/H2D work for norm/attention/routing, per-token GEMM loops, and per-route MoE replay.

Could you open a follow-up issue for that scheduler/runtime profiling work and link it from this benchmark snapshot?

[CAICAIIs]

Thanks for preserving this matrix. One thing the retained results seem to highlight is a serving-scheduler/runtime follow-up rather than a benchmark-script issue: OpenInfer reaches active request sets of 4/8, but output throughput stays almost flat while vLLM TP2/TP2+EP2 scales with concurrency.

From the current trace fields, it would be useful to track why active requests are not turning into effective decode throughput. The likely areas to profile are: serial per-request prefill/admission, same-position-only decode subgrouping, per-layer host D2H/H2D work for norm/attention/routing, per-token GEMM loops, and per-route MoE replay.

Could you open a follow-up issue for that scheduler/runtime profiling work and link it from this benchmark snapshot?

got it, done