|
| 1 | +# Project Summary |
| 2 | + |
| 3 | +**Agentic Kernel Generation** is an LLM-agent system that writes, verifies, |
| 4 | +and evolves Triton GPU kernels for LLM-inference operators — with a benchmark |
| 5 | +harness designed so the reported speedups can't be faked. |
| 6 | + |
| 7 | +## The idea |
| 8 | + |
| 9 | +An LLM generates a Triton kernel. A sandboxed harness checks it against a |
| 10 | +PyTorch reference on unseen random inputs and profiles it. A reflection agent |
| 11 | +turns the profile into the next optimization, and an evolutionary controller |
| 12 | +breeds the best kernels across generations. |
| 13 | + |
| 14 | +## What makes it different |
| 15 | + |
| 16 | +- **Reward-hack-proof measurement.** Correctness is a hard gate, the kernel |
| 17 | + never sees the reference output, and every timed iteration uses fresh inputs. |
| 18 | +- **Learns from its own failures.** A knowledge base distilled from 1,000+ real |
| 19 | + Triton errors feeds verified fixes back into the refine loop. |
| 20 | + |
| 21 | +## Results |
| 22 | + |
| 23 | +Runs on a single 8 GB laptop GPU (RTX 3070, Ampere). Highlights vs PyTorch eager: |
| 24 | +sliding-window attention 174.7×, W4A16 matmul 24.5×, flash attention 11.5× — |
| 25 | +and ahead of PyTorch's own fused kernels (FlexAttention 1.48×, SDPA 1.17×) on |
| 26 | +selected workloads. |
| 27 | + |
| 28 | +## Structure |
| 29 | + |
| 30 | +Two nested loops: an inner loop that refines one candidate serially |
| 31 | +(evaluate → reflect → refine), and an outer evolutionary loop that selects |
| 32 | +parents from a MAP-Elites archive and breeds offspring by mutation or crossover. |
| 33 | +Fitness is speedup, hard-gated to zero unless the kernel is correct. |
| 34 | + |
| 35 | +See the [README](README.md) for full results, methodology, and quickstart. |
0 commit comments