Skip to content

Commit ca43055

Browse files
committed
refactor: consolidate benchmark presentation into benchmarks/presentation/; widen charts, drop eyebrows
1 parent c362e51 commit ca43055

12 files changed

Lines changed: 9197 additions & 41 deletions

File tree

.gitignore

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,8 @@ build/
3131
.clawpatch/
3232

3333
# generated benchmark reports
34-
report/
34+
benchmarks/presentation/review.html
35+
benchmarks/presentation/scoreboard.html
3536
.playwright-mcp/
3637

3738
# tuner: cached propose artifacts + captured regressions (smoke.seed.json + baselines/ stay committed)

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -235,7 +235,7 @@ output (Impeccable, Codex, Claude, and more) converts to native blocks, across s
235235
converters (the built-in rules, plus experimental LLM translators run via their CLIs).
236236

237237
```sh
238-
npm run bench # score the suite; write report/review.html + report/scoreboard.html
238+
npm run bench # score the suite; write benchmarks/presentation/review.html + benchmarks/presentation/scoreboard.html
239239
npm run bench:record # also append a provenance-tagged run to benchmarks/results.jsonl
240240
```
241241

benchmarks/README.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -12,11 +12,11 @@ npm run bench
1212
```
1313

1414
This prints a scorecard (overall + per-producer) and writes two generated pages (gitignored,
15-
segregated from the committed suite) under `report/`:
15+
segregated from the committed suite) under `benchmarks/presentation/`:
1616

17-
- **`report/review.html`** — per layout: the ideal end state beside *each producer's* render
17+
- **`benchmarks/presentation/review.html`** — per layout: the ideal end state beside *each producer's* render
1818
(the cross-producer comparison matrix).
19-
- **`report/scoreboard.html`** — scores over time, built from the results log.
19+
- **`benchmarks/presentation/scoreboard.html`** — scores over time, built from the results log.
2020

2121
```sh
2222
npm run bench:record # same run, but appends one record to benchmarks/results.jsonl
@@ -250,7 +250,7 @@ Three additive extension points — none requires touching the core:
250250
2. **Engine** (a converter) — `scripts/engines/<name>.ts` exporting `convert()`.
251251
3. **Output target** (a block vocabulary, e.g. CoBlocks) — an adapter; directional, see `md/04` + `md/09`.
252252

253-
Run `npm run bench` (prints the scorecard + writes `report/`); `npm run bench:record`
253+
Run `npm run bench` (prints the scorecard + writes `benchmarks/presentation/`); `npm run bench:record`
254254
appends a provenance-tagged run to `results.jsonl`. The golden rule (`md/11`): **be
255255
specific about the output (core blocks), neutral about the variable axes — never
256256
special-case one producer or one engine/model.** The `suiteHash` tells you when two runs
Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,16 @@
1-
# Benchmark visuals
1+
# Benchmark presentation
22

33
Public, screenshot-ready visualisations for the README and other collateral. One
4-
self-contained `index.html` + one lightweight `style.css` (Geist, no build, no
4+
self-contained `figures.html` + one lightweight `style.css` (Geist, no build, no
55
dependencies).
66

77
## View
88

9-
Open `index.html` directly, or serve the folder:
9+
Open `figures.html` directly, or serve the folder:
1010

1111
```sh
12-
cd visuals && python3 -m http.server 8799
13-
# http://localhost:8799/index.html
12+
cd benchmarks/presentation && python3 -m http.server 8799
13+
# http://localhost:8799/figures.html
1414
```
1515

1616
Then screenshot each figure card (or the whole page) and drop the PNGs into the
@@ -25,10 +25,13 @@ main `README.md`.
2525

2626
Figures 1–2 are data-driven; 3–4 are static diagrams.
2727

28+
The generated `review.html` + `scoreboard.html` now live in this same folder
29+
(gitignored, regenerated by `npm run bench`).
30+
2831
## Refreshing the data
2932

3033
The numbers are embedded as `MODELS` and `RUNS` constants in a `<script>` at the
31-
bottom of `index.html`, transcribed from `benchmarks/results.jsonl`. When new runs
34+
bottom of `figures.html`, transcribed from `benchmarks/results.jsonl`. When new runs
3235
land, update those two constants and re-screenshot — the charts scale to any number
3336
of runs automatically.
3437

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,6 @@
1010
<div class="wrap">
1111

1212
<header class="page-head">
13-
<p class="eyebrow">Block Runner · Conversion benchmark</p>
1413
<h1>How faithfully does HTML become native blocks?</h1>
1514
<p class="lede">Every conversion is scored against a fixed suite of design sections with a known ideal block
1615
tree. Higher means the output matched the intended <code style="font-family:var(--font-mono)">wp:*</code>
@@ -27,7 +26,6 @@ <h1>How faithfully does HTML become native blocks?</h1>
2726
<!-- HEADLINE: value by complexity -->
2827
<section class="figure">
2928
<div class="figure-head">
30-
<p class="eyebrow">The result</p>
3129
<h2>Raw LLM vs Block Runner, by block complexity</h2>
3230
<p class="sub">The same conversion task scored five ways — two raw LLMs writing the markup themselves (no Block
3331
Runner), and three Block Runner paths: rules only, then a model (GPT-5.5 / Opus) proposing the structure
@@ -47,7 +45,6 @@ <h2>Raw LLM vs Block Runner, by block complexity</h2>
4745
<!-- DETAIL: per-block dots -->
4846
<section class="figure">
4947
<div class="figure-head">
50-
<p class="eyebrow">The result · per block</p>
5148
<h2>Every block tested, as one dot</h2>
5249
<p class="sub">The same five paths and the same Simple / Complex split — but each dot is one block’s score,
5350
so you can see the spread the averages hide. The bar above each path is its mean.</p>
@@ -61,7 +58,6 @@ <h2>Every block tested, as one dot</h2>
6158
<!-- FIGURE E -->
6259
<section class="figure">
6360
<div class="figure-head">
64-
<p class="eyebrow">How it works</p>
6561
<h2>The path from any HTML to clean native blocks</h2>
6662
<p class="sub">What runs when an end user pipes HTML through the CLI. One pass: translate the structure,
6763
prove it against real Gutenberg, fill in real media — and surface anything it couldn’t express natively.</p>
@@ -75,7 +71,6 @@ <h2>The path from any HTML to clean native blocks</h2>
7571
<!-- FIGURE F -->
7672
<section class="figure">
7773
<div class="figure-head">
78-
<p class="eyebrow">How it’s measured</p>
7974
<h2>The benchmark loop</h2>
8075
<p class="sub">One ideal answer per layout; many producers supply HTML; the engine converts; a deterministic
8176
scorer grades the result against the ideal tree; every run is logged with provenance so progress is auditable.</p>
File renamed without changes.
Lines changed: 4 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -60,20 +60,12 @@ body {
6060
}
6161

6262
.wrap {
63-
max-width: 980px;
63+
max-width: 1280px;
6464
margin: 0 auto;
65-
padding: 56px 28px 80px;
65+
padding: 56px 44px 80px;
6666
}
6767

6868
/* ---- header ---- */
69-
.eyebrow {
70-
font-family: var(--font-mono);
71-
font-size: 11.5px;
72-
letter-spacing: 0.14em;
73-
text-transform: uppercase;
74-
color: var(--faint);
75-
margin: 0 0 12px;
76-
}
7769
.page-head h1 {
7870
font-size: 34px;
7971
line-height: 1.1;
@@ -106,8 +98,8 @@ body {
10698
border: 1px solid var(--line);
10799
border-radius: 16px;
108100
box-shadow: var(--shadow);
109-
padding: 26px 28px 22px;
110-
margin: 28px 0;
101+
padding: 40px 44px 34px;
102+
margin: 40px 0;
111103
}
112104
.figure-head { margin-bottom: 8px; }
113105
.figure-head h2 {

0 commit comments

Comments
 (0)