Skip to content

Commit 29c7a33

Browse files
committed
Refactor attention subsystem into 4-layer Onion Architecture (Phases 1-6)
- Extract legacy UNet transformer blocks from attention_flax.py into src/maxdiffusion/models/unet_transformer_blocks_flax.py (-68% LOC reduction in attention_flax.py). - Create src/maxdiffusion/models/attention_strategies/ package with communication-only strategies (SingleShardStrategy, RingAttentionStrategy, UlyssesStrategy, CustomRingAttentionStrategy) and Protocol definition. - Introduce extensible KERNEL_REGISTRY in attention_utils.py and attention_dispatch.py with @register_kernel decorators for all 13 attention backends. - Implement robust memory-efficient attention chunking and ValueError tensor validation in attention_dispatch.py. - Implement scan-based ring communication loops (length=ring_size-1) without static loop unrolling in custom_ring.py to keep compile times and HLO graph size minimal. - Ensure exact cross-attention sequence length and value head dimension handling across Ulysses and Ring strategies. - Maintain 100% mathematical and operational parity with origin/main across all attention kernels (dot_product, Pallas splash, TokaMax ring, Ulysses, and custom dense splash kernels). - Add unit test suite src/maxdiffusion/tests/attention_strategies_test.py covering strategy instantiation, hooks, block size adapters, and kernel registry validation.
1 parent df69836 commit 29c7a33

17 files changed

Lines changed: 3967 additions & 3054 deletions

src/maxdiffusion/kernels/custom_splash_attention.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,8 @@ def __init__(
6363
# logits differently, breaking the cross-shard merge). Without k-smoothing the
6464
# per-row max logit has no >=0 guarantee, so the safe bound halves (calibrated
6565
# for ring_size=2, matching DiffusionServing's ring gate).
66-
_FIXED_M_RING_SAFE_BOUND = _FIXED_M_SAFE_BOUND / 2.0
66+
FIXED_M_RING_SAFE_BOUND = _FIXED_M_SAFE_BOUND / 2.0
67+
_FIXED_M_RING_SAFE_BOUND = FIXED_M_RING_SAFE_BOUND
6768

6869

6970
def _flash_attention_kernel(
@@ -815,3 +816,9 @@ def _splash_attention(q, k, v, mk=None):
815816
)
816817

817818
return _splash_attention
819+
820+
821+
# Public aliases for first-class attention strategies
822+
BlockSizes = _BlockSizes
823+
FIXED_M_SAFE_BOUND = _FIXED_M_SAFE_BOUND
824+
splash_attention_forward_ring = _splash_attention_forward_ring

src/maxdiffusion/kernels/splash_attention/ring_attention_kernel.py

Lines changed: 7 additions & 463 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)