Skip to content

feat(powerx): annotate measurement window and reconcile trace vs published power | PowerX:标注测量窗口并对账原始轨迹与已发布功耗 - #940

Open
edwingao28 wants to merge 9 commits into
masterfrom
feat/gpu-power-window-reconciliation
Open

feat(powerx): annotate measurement window and reconcile trace vs published power | PowerX:标注测量窗口并对账原始轨迹与已发布功耗#940
edwingao28 wants to merge 9 commits into
masterfrom
feat/gpu-power-window-reconciliation

Conversation

@edwingao28

@edwingao28 edwingao28 commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

The hidden PowerX viewer (/gpu-metrics) rendered raw GPU telemetry with no indication of which part of the trace was formally measured, whether the measurement was declared valid, or whether the published avg_power_w matches the trace.

  • GET /api/gpu-metrics (additive): each gpu_metrics_<X> artifact entry can now carry an optional power block assembled server-side from same-suffix sibling artifacts, cheapest first — the tiny bmk_<X> / bmk_agentic_<X> agg row (Tier 1; sufficient once the row-level power provenance change queued in InferenceX (feat/power-row-provenance) lands its power_audit object there), falling back to the power_audit_<X> bundle's power_validation sidecar (Tier 2, same 50 MB Content-Length cap as the CSV path). Tolerant in both directions: agg rows from before and after feat/power-row-provenance, single-node benchmark_window and multinode selected_window sidecars, and the agentic results/power_validation.json entry naming introduced by the AgentX power artifacts change queued in InferenceX (feat/agentx-power-artifacts) all map through the same pure helpers (src/lib/power-audit-artifacts.ts).
  • Chart: a new custom D3 layer shades the measurement window with dashed bounds and labels, dims warmup/post-benchmark regions, and tracks x-zoom.
  • Reconciliation panel: validity badge + verbatim reason chips, window bounds/duration, observed vs expected chip counts, published avg_power_w (with source), a viewer-side trapezoid recompute over all chips (mirroring the producer's boundary-interpolated per-device integration), and a color-coded delta (|Δ| ≤ 2% ok, ≤ 5% warn — the producer's accumulator tolerance — else alert), plus provenance (producer sha / exporter digest) when present.
  • Timestamp alignment: naive nvidia-smi/ISO trace timestamps are placed via UTC parsing; a window that cannot be intersected with the trace yields an explicit message and no shading — never a silently shifted band.

Backward compatibility

  • Runs with no sidecar artifacts (or oversized/malformed ones) return a response shape-identical to master — no power key, CSV view untouched; a sidecar failure can never break the trace view.
  • No power block → no panel, no shading: the page renders bit-for-bit as before (covered by an intercepted legacy-degradation e2e spec).
  • /api/gpu-metrics stays classified ui-artifact-read (excluded from the stable v1 catalog); the review digest was refreshed.

Tests

  • src/lib/power-audit-artifacts.test.ts (new): suffix derivation, agg mapping for both legacy and feat/power-row-provenance row shapes, single-node/multinode sidecar mapping, implausible-epoch rejection, deterministic validation-entry selection, merge precedence.
  • src/components/gpu-power/power-window.test.ts (new): TZ-independent UTC parsing, window alignment (inside / clamped / disjoint-null), hand-computed trapezoid integration (constant, ramp, boundary interpolation, mixed cadences), delta thresholds.
  • src/app/api/gpu-metrics/route.test.ts: every pre-existing case unmodified; new cases for sidecar-only runs, Tier-1 short-circuit (asserted via fetch call count), sibling-free legacy shape, oversized bundles, malformed sidecar JSON, and no-op bmk source attribution.
  • cypress/e2e/gpu-power.cy.ts: two new fixture-driven specs (token-free via cy.intercept) — window shading + reconciliation (published 402 W vs recomputed 400 W → Δ −0.5%) and legacy degradation.

Full app unit suite: 4514 passed (254 files). bunx tsc --noEmit: clean. gpu-power.cy.ts: 11/11 passed against the E2E_FIXTURES=1 dev server.

Review notes

The following issues were found during review and fixed in fe82c574:

  • Corrupt sidecar epoch could crash the page: a finite-but-out-of-range start_time_unix (e.g. 1e16) reached new Date(v * 1000).toISOString() in the panel and threw RangeError. Fixed at both ends: windowFromUnixPair now rejects non-positive, ms-scale, and past-2100 epochs (unit-tested), and the panel formats bounds through a helper that falls back to the raw number outside Date's representable range.
  • sources footer overstated provenance: a bmk_* artifact whose agg row carried no power fields was still credited, and a bundle agg row contributing only window/counts wasn't. Artifacts are now credited only when their parsed content contributed (hasPowerContent gate); a no-op Tier-1 row also falls through to the Tier-2 agg fallback (new route test).
  • Zooming fully into warmup/post lost the dim overlay: the degenerate-band early return dropped all shading, making a fully-zoomed warmup region look like measured data. The layer now dims the whole visible region (with the warmup/post label) when the view lies wholly on one side of the window.
  • Sidecar schema_version leaked into published.power_metric_schema_version: the sidecar's own schema version (1) is a different versioning axis than the agg row's power-metric schema version (2). The validation_metrics fallback now reports null.

One residual limitation is deliberate: a non-UTC runner with a clock offset smaller than the trace span would still intersect and shade a shifted window. Heuristic timezone snapping was rejected in favor of the intersection-check-or-refuse design, and benchmark containers run UTC in practice; this is documented as a known residual risk rather than guessed at.

Out of scope

Multinode traces (no gpu_metrics_* artifact exists to attach to), DB-vs-artifact drift checks (InferenceX-app PR #939), and AgentX artifact publication (feat/agentx-power-artifacts — once it uploads gpu_metrics_<X>/power_audit_<X> for agentic lanes, this viewer lights up with zero app changes).


Note

Medium Risk
Adds GitHub artifact downloads and JSON parsing on a live UI route; failures are guarded but wrong window alignment could mislead reviewers of benchmark power.

Overview
PowerX now surfaces formal benchmark power metadata alongside raw GPU traces: validity, measurement window, published vs viewer-recomputed average power, and chart shading for warmup vs measured regions.

GET /api/gpu-metrics optionally attaches a normalized power object per gpu_metrics_* artifact by reading same-suffix bmk_* / power_audit_* ZIPs (prefer agg row; fall back to validation sidecar when window bounds are missing). Sidecar failures are isolated so CSV-only responses stay unchanged.

The UI adds a reconciliation card (verdict, window, chip counts, published vs trapezoid recompute, delta bands) and extends the line chart with a measurement-window band, dashed bounds, and dimmed warmup/post regions that follow x-zoom.

New pure helpers live in power-audit-artifacts.ts and power-window.ts, with unit tests, route tests, a Cypress fixture, and three new e2e cases (full flow, window-less power, legacy without power).

Reviewed by Cursor Bugbot for commit ed6294c. Bugbot is set up for automated code reviews on this repo. Configure here.

…:在 /api/gpu-metrics 上公开功耗审计侧车数据块

For each gpu_metrics_<X> artifact, assemble an optional additive `power`
block from same-suffix siblings, cheapest first: the tiny bmk_<X> /
bmk_agentic_<X> agg row (Tier 1, sufficient once its PLAN-06 power_audit
object carries window bounds), then the power_audit_<X> bundle's
power_validation sidecar (Tier 2, same 50 MB cap as the CSV path).
Every step tolerates absence, oversize, and malformed JSON — legacy runs
return a byte-compatible response with no `power` key, and a sidecar
failure never breaks the CSV view.

为每个 gpu_metrics_<X> 产物按同后缀兄弟产物组装可选的附加 `power` 数据块,
优先最便宜的来源:先取轻量的 bmk_<X>/bmk_agentic_<X> 聚合行(第 1 层,一旦
其 PLAN-06 power_audit 对象携带窗口边界即可满足),否则再下载
power_audit_<X> 捆绑包中的 power_validation 侧车(第 2 层,与 CSV 路径共用
50 MB 上限)。每一步都容忍缺失、超限与损坏的 JSON —— 历史运行返回与现状
字节兼容的响应(无 `power` 键),侧车失败绝不影响 CSV 视图。
…lished power | PowerX:标注测量窗口并对账原始轨迹与已发布功耗

The chart gains a custom D3 layer that shades the formal measurement
window with dashed bounds and dims warmup/post-benchmark regions,
tracking x-zoom. A reconciliation panel shows the validity verdict with
verbatim reason chips, window bounds, GPU counts, the published
avg_power_w, and a viewer-side trapezoid recompute over all chips
(mirroring the producer's boundary-interpolated integration), with a
color-coded delta (|Δ| ≤ 2% ok, ≤ 5% warn, else alert). Naive trace
timestamps are placed via UTC parsing; a window that cannot be aligned
yields an explicit message and no shading — never a silently shifted
band. Runs without a power block render exactly as before.

图表新增自定义 D3 图层:以虚线边界标注正式测量窗口、并将预热/基准结束后
区域调暗,且跟随横向缩放。新增对账面板展示有效性结论(原样显示原因代
码)、窗口边界、芯片数量、已发布 avg_power_w,以及查看器端对全部芯片的
梯形积分重算(镜像生产者的边界插值积分),偏差按 |Δ| ≤ 2% 正常、≤ 5%
警告、否则告警着色。无时区的轨迹时间戳按 UTC 解析定位;无法对齐的窗口
会明确提示且不绘制阴影 —— 绝不静默偏移。没有 power 数据块的运行渲染与
之前完全一致。
…ation | 测试:为窗口标注与旧数据降级添加基于夹具的端到端用例

Adds a synthetic gpu-metrics run fixture (2 chips, 100 epoch-timestamp
samples each, constant 400 W, published 402 W → Δ −0.5%) registered in
the fixture manifest, plus two intercepted specs: the enriched response
must shade the window and reconcile, and a power-less response must
render the page exactly as before (no panel, no shading). Both run
token-free via cy.intercept.

新增合成的 gpu-metrics 运行夹具(2 块芯片、各 100 个 epoch 时间戳采样、
恒定 400 W,已发布 402 W → 偏差 −0.5%)并登记到夹具清单,另加两个拦截
式用例:增强响应必须标注窗口并完成对账;无 power 字段的响应必须与之前
完全一致地渲染页面(无面板、无阴影)。两者均通过 cy.intercept 免令牌
运行。
…杂项:格式化后刷新 /api/gpu-metrics 路由摘要

The pre-commit formatter rewrote route.ts after the review digest was
recorded; same contract, same ui-artifact-read classification.

预提交格式化在记录审查摘要后重写了 route.ts;契约与 ui-artifact-read
分类均未变化。
… overlay | PowerX:加固窗口时间戳校验、来源脚注与缩放后的置灰遮罩

Review fixes for #931:
- reject implausible unix-second window bounds (garbage/ms-scale epochs)
  in windowFromUnixPair and format window bounds defensively, so a corrupt
  sidecar can never crash the page with a Date RangeError
- credit an artifact in the power block's sources footer only when its
  parsed content actually contributed (no-op bmk agg rows fall through to
  the Tier-2 bundle agg fallback)
- keep the warmup/post-benchmark dim overlay when the chart is zoomed
  wholly outside the measurement window instead of dropping all shading
- stop mapping the validation sidecar's own schema_version into
  published.power_metric_schema_version (different versioning axis)
@edwingao28
edwingao28 requested a review from adibarra as a code owner August 28, 2026 02:57
@vercel

vercel Bot commented Aug 28, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
inferencemax-app Ready Ready Preview Aug 31, 2026 9:13pm

Request Review

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Claude finished @edwingao28's task in 3m 32s —— View job


Review: PowerX window reconciliation — 🟢 LGTM

  • Read the diff & new files
  • Review artifact-mapping helpers (power-audit-artifacts.ts)
  • Review window math (power-window.ts)
  • Review API route changes
  • Review chart + reconciliation UI
  • Check CI status
  • Runtime verification (localhost)
  • Post findings

No blocking issues. This is a careful, well-tested change; the two prior review passes already caught and fixed the realistic failure modes (corrupt-epoch RangeError, provenance over-crediting, lost dim-overlay on deep zoom).

What I verified

  • Runtime: /gpu-metrics renders cleanly with 0 console errors (feature gate open in this env). The new power block can't be exercised against a live run because the sidecar artifacts it reads don't exist in production yet (PLAN-06 pending) — which is exactly why the PR ships fixture-driven e2e (gpu-power.cy.ts) plus a legacy "no power key" degradation spec. Backward-compat render path confirmed intact.
  • CI: Lint & Format ✅, Tests (Unit) ✅ (E2E still in progress at review time). Fold in the E2E result before merge.
  • Math correctness: window bounds and trace samples are both compared in absolute UTC seconds (window.start_unix vs ms/1000), so the trapezoid recompute and the producer's per-device integration are on the same clock. Boundary interpolation (interpolatePower) is only ever called with clamped in-range boundaries (deviceStart ≥ samples[0], deviceEnd ≤ last), so no out-of-bounds destructure. ?? is used correctly around the falsy-but-valid power_valid: 0. formatWindowBound guards Date's representable range.
  • Additive contract: a sidecar failure is caught per-artifact and can never break the CSV trace view; hasPowerContent gating keeps the sources footer and Tier-1→Tier-2 fallback honest.
  • Tests present for all new lib/UI code (power-audit-artifacts.test.ts, power-window.test.ts, expanded route.test.ts, gpu-power.cy.ts) — no test-coverage gap.

Non-blocking observation (no action required)

  • The chart's data path (GpuPowerChart.buildGroupedData) parses naive nvidia-smi timestamps with local new Date(), while window alignment (power-window.parseTimestampUtcMs) parses them as UTC. As the code comments correctly note, this only shifts every sample by a constant, so relative-seconds agree — except across a viewer-timezone DST boundary within a single trace, where the local offset isn't constant and the shaded band could sit off by up to the DST delta for post-transition samples. Impact is negligible (traces are minutes-long, never a crash), and the intersection-or-refuse design already documents the UTC-clock assumption as a residual risk. Flagging only for the record.

Nice work — the reconciliation panel, delta thresholds tied to the producer's tolerance, and the intersection-check-or-refuse alignment are all sound.
· feat/gpu-power-window-reconciliation

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit eb9868a. Configure here.

Comment thread packages/app/src/components/gpu-power/GpuPowerDisplay.tsx Outdated
中文:清理 PowerX 对账代码中的内部编号、重复说明和模糊注释
中文:更新 GPU 功耗路由摘要并明确审计窗口测试名称
中文:明确 power_audit 为聚合结果中的嵌入式审计对象
中文:在缺少测量窗口边界时仍显示已发布平均功耗
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant