Skip to content

Commit 46dbac3

Browse files
Mossakaclaude
andauthored
perf: bump benchmark iterations from 5 to 30 and wire up workflow input (#1870)
With only 5 iterations, P95/P99/max are all the same value — no statistical significance in percentile calculations. Bumping to 30 gives meaningful tail-latency data (P95 = 2nd-worst of 30 runs). Changes: - Read iteration count from AWF_BENCHMARK_ITERATIONS env var (default 30) - Pass workflow_dispatch iterations input to the script via env var - Increase workflow timeout from 30 to 90 minutes for more iterations Closes #1864 Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 71f2e50 commit 46dbac3

2 files changed

Lines changed: 5 additions & 3 deletions

File tree

.github/workflows/performance-monitor.yml

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ on:
99
iterations:
1010
description: "Number of iterations per metric"
1111
required: false
12-
default: "5"
12+
default: "30"
1313

1414
permissions:
1515
contents: read
@@ -19,7 +19,7 @@ jobs:
1919
benchmark:
2020
name: Run Performance Benchmarks
2121
runs-on: ubuntu-latest
22-
timeout-minutes: 30
22+
timeout-minutes: 90
2323
steps:
2424
- name: Checkout
2525
uses: actions/checkout@v4
@@ -46,6 +46,8 @@ jobs:
4646
4747
- name: Run benchmarks
4848
id: benchmark
49+
env:
50+
AWF_BENCHMARK_ITERATIONS: ${{ github.event.inputs.iterations || '30' }}
4951
run: |
5052
npx tsx scripts/ci/benchmark-performance.ts > benchmark-results.json 2>benchmark-progress.log || true
5153
cat benchmark-progress.log

scripts/ci/benchmark-performance.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ import { stats, parseMb, checkRegressions, BenchmarkResult, BenchmarkReport } fr
2020

2121
// ── Configuration ──────────────────────────────────────────────────
2222

23-
const ITERATIONS = 5;
23+
const ITERATIONS = parseInt(process.env.AWF_BENCHMARK_ITERATIONS || '30', 10);
2424
const AWF_CMD = "sudo awf";
2525
const ALLOWED_DOMAIN = "api.github.com";
2626
const CLEANUP_CMD = "sudo docker compose down -v 2>/dev/null; sudo docker rm -f awf-squid awf-agent 2>/dev/null; sudo docker network prune -f 2>/dev/null";

0 commit comments

Comments
 (0)