Minimal FlashAttention in CUDA C++/CuTe: readable WMMA/CuTe kernels, no NxN workspace, up to 4.5x faster than naive PyTorch
High-performance grouped matrix multiplication for fine-grained and ultra-fine-grained MoE workloads.
16-step CUDA optimization of FlashAttention-2 achieving 99.2% of official performance on A100 — Ampere architecture
Production-grade FlashAttention FP8 e4m3 forward kernel for NVIDIA Blackwell consumer GPUs (sm_120a, e.g. RTX PRO 6000). 647–652 TFLOPS at hd=128, sl=8192. Multi-kernel dispatcher, C library with Go and Python bindings
🍭 Sweet GPU compute kernels in CUDA, wrapped via CuPy
CUDA reduction kernels benchmarked against CUB, with optimization experiments across multiple implementation strategies.
what I tried first, where it broke, and why I ended up where I did. If you just want the headline results, they're in the "Where I landed" sections. Everything else here is the reasoning behind them. The raw code, logs, and profiler screenshots referenced below are in the Task_1/ and Task_2/ folders (see the map at the bottom).
Add a description, image, and links to the gpu-kernels topic page so that developers can more easily learn about it.
To associate your repository with the gpu-kernels topic, visit your repo's landing page and select "manage topics."