Skip to content

Commit 7d1b25d

Browse files
committed
Refactor attention subsystem into 4-layer Onion Architecture (Phases 1-6)
- Extract legacy UNet transformer blocks from attention_flax.py into src/maxdiffusion/models/unet_transformer_blocks_flax.py (-68% LOC reduction in attention_flax.py). - Create src/maxdiffusion/models/attention_strategies/ package with communication-only strategies (SingleShardStrategy, RingAttentionStrategy, UlyssesStrategy, CustomRingAttentionStrategy) and Protocol definition. - Introduce extensible KERNEL_REGISTRY in attention_utils.py and attention_dispatch.py with @register_kernel decorators for all 13 attention backends. - Implement robust memory-efficient attention chunking and ValueError tensor validation in attention_dispatch.py. - Implement scan-based ring communication loops (length=ring_size-1) without static loop unrolling in custom_ring.py to keep compile times and HLO graph size minimal. - Ensure exact cross-attention sequence length and value head dimension handling across Ulysses and Ring strategies. - Maintain 100% mathematical and operational parity with origin/main across all attention kernels (dot_product, Pallas splash, TokaMax ring, Ulysses, and custom dense splash kernels). - Add unit test suite src/maxdiffusion/tests/attention_strategies_test.py covering strategy instantiation, hooks, block size adapters, and kernel registry validation.
1 parent df69836 commit 7d1b25d

18 files changed

Lines changed: 3989 additions & 3055 deletions

src/maxdiffusion/kernels/custom_splash_attention.py

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,22 @@ def __init__(
4646
self.block_kv_compute = block_kv_compute if block_kv_compute is not None else block_kv
4747
self.block_kv_compute_in = block_kv_compute_in
4848

49+
@property
50+
def bq(self) -> int:
51+
return self.block_q
52+
53+
@property
54+
def bkv(self) -> int:
55+
return self.block_kv
56+
57+
@property
58+
def bkv_compute(self) -> int:
59+
return self.block_kv_compute
60+
61+
@property
62+
def bkv_compute_in(self) -> int:
63+
return self.block_kv_compute_in
64+
4965

5066
# Fixed-m softmax-bound constants. Instead of tracking the online-softmax
5167
# running max per KV block, eligible heads subtract a precomputed per-query
@@ -63,7 +79,8 @@ def __init__(
6379
# logits differently, breaking the cross-shard merge). Without k-smoothing the
6480
# per-row max logit has no >=0 guarantee, so the safe bound halves (calibrated
6581
# for ring_size=2, matching DiffusionServing's ring gate).
66-
_FIXED_M_RING_SAFE_BOUND = _FIXED_M_SAFE_BOUND / 2.0
82+
FIXED_M_RING_SAFE_BOUND = _FIXED_M_SAFE_BOUND / 2.0
83+
_FIXED_M_RING_SAFE_BOUND = FIXED_M_RING_SAFE_BOUND
6784

6885

6986
def _flash_attention_kernel(
@@ -815,3 +832,9 @@ def _splash_attention(q, k, v, mk=None):
815832
)
816833

817834
return _splash_attention
835+
836+
837+
# Public aliases for first-class attention strategies
838+
BlockSizes = _BlockSizes
839+
FIXED_M_SAFE_BOUND = _FIXED_M_SAFE_BOUND
840+
splash_attention_forward_ring = _splash_attention_forward_ring

src/maxdiffusion/kernels/splash_attention/ring_attention_kernel.py

Lines changed: 7 additions & 463 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)