Skip to content

e2e-real: perf recording must judge before it writes — EPYC regressions are currently recorded, not gated #214

Description

@VijitSingh97

Why

The release perf gate (e2e-real.sh perf) only ever judges on miner-0 — the pre-tag gate rig. The seven EPYCs meet the perf phase exactly once per release, post-deploy, with E2E_PERF_RECORD=1 — and record mode returns right after writing the new baseline + history entry, skipping the baseline and best-ever checks entirely. A class-wide EPYC regression (say a codegen change that costs 4% on Zen 2 but nothing on Zen 4) would be silently enshrined as the new baseline, and the anti-ratchet's best-ever comparison never runs on the hosts where it matters most. 7/8 of the fleet's hashrate is effectively un-gated per release.

Scope

  • Record mode judges first. With E2E_PERF_RECORD=1 and an existing baseline/history: measure, run the same baseline + best-ever checks as judge mode, and on a within-tolerance result proceed to write (one bench serves both purposes — no double 1-minute run).
  • A regression fails the phase and writes nothing — a regressed number must never become the new baseline by default. E2E_PERF_FORCE=1 is the conscious override for the documented "investigate or deliberately re-record" flow (writes AND says loudly that it recorded a regression).
  • First-ever recording (no baseline yet) records as today.
  • RELEASING.md's post-deploy step gains one sentence: the per-rig recording is also the per-rig gate — a failed rig means investigate before publishing the fleet as healthy.

Acceptance criteria

  • E2E_PERF_RECORD=1 with a healthy measurement: judged ok + recorded (both asserted).
  • With a measurement below tolerance vs baseline or best-ever: phase FAILS, baseline/history untouched; E2E_PERF_FORCE=1 records it with a loud warning.
  • No baseline on disk: records without judgment (unchanged).
  • make lint + make test green; RELEASING.md updated.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    performanceHashrate / tuning / efficiencytestingTests, CI, and test infrastructure

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions