feat(anima) - Anima Performance Improvements

[kappacommit]

Summary

Two small, self-contained performance fixes for Anima generation, mainly benefiting GPUs with limited VRAM
(e.g. 8 GB).

1. Tiled VAE decode (anima_l2i)
   The Anima VAE decode now uses tiling, which keeps its memory use bounded. Previously a 1024×1024 decode could
   spike close to the VRAM limit and take anywhere from 2 to 11 seconds; with tiling it's a consistent ~1–2
   seconds. Images 512px and smaller are unaffected — they still decode in a single pass.

2. cuDNN attention (anima_denoise, Windows + CUDA only)
   On Windows, PyTorch doesn't ship its fastest attention kernel, so the denoiser was defaulting to a noticeably
   slower one. This opts the denoiser into cuDNN's attention kernel — roughly 1.7× faster for Anima — the same
   thing ComfyUI does. It's a prioritized preference, not a hard requirement: torch falls back automatically on
   any GPU where cuDNN isn't available, and it's scoped to Windows + CUDA where it's actually needed.

Anima performance analysis

Three configurations, all measured with the same per-generation instrumentation
(setup = model load + LLM-adapter, denoise = sampling loop, VAE = decode):

• fixes-off — both code fixes disabled (baseline)
• fixes-on — cuDNN flash attention + tiled VAE decode
• fixes-on + cache — the above, plus max_cache_ram_gb: 16

Setup: RTX 4070 Laptop (8 GB VRAM) · 30 steps · Euler · CFG. Figures are
warm steady-state averages — the cold first run of each batch is excluded
(see Notes). Warm runs per cell: 4 (fixes-off) / 9 (1024²) / 4 (1280²).

1024×1024

Config	setup (s)	denoise (s)	s/it	VAE decode (s)	Total (s)
fixes-off	0.29	52.9	1.76	6.4	59.3
fixes-on	0.26	47.9	1.60	1.1	49.0
fixes-on + cache	0.25	48.2	1.61	1.1	49.3

1280×1280

Config	setup (s)	denoise (s)	s/it	VAE decode (s)	Total (s)
fixes-off	0.91	86.2	2.87	8.5	94.7
fixes-on	0.74	75.7	2.52	1.8	77.5
fixes-on + cache	0.67	75.3	2.51	1.8	77.1

Findings

1. The code fixes — clear, significant win

	1024²	1280²
Denoise — cuDNN attention	52.9 → 47.9 s (−9 %)	86.2 → 75.7 s (−12 %)
VAE decode — tiling	6.4 → 1.1 s (−84 %)	8.5 → 1.8 s (−78 %)
Total	59.3 → 49.0 s (−17 %)	94.7 → 77.5 s (−18 %)

• cuDNN attention delivers a steady per-step gain: 1.76 → 1.60 s/it (1024²),
  2.87 → 2.52 s/it (1280²).
• VAE tiling is the larger effect here. In this run the untiled decode was
  consistently slow (6–9 s) because VRAM was under pressure — the full decode
  spikes badly when memory is tight. Across all baseline runs the untiled
  decode ranged 0.7–9.7 s; the tiled decode is a steady ~1.1 s / ~1.8 s.
  The win is speed and, more importantly, predictability.

2. The RAM-cache bump — no measurable effect (in this test)

fixes-on vs fixes-on + cache is a wash at both resolutions — 49.0 vs 49.3 s,
77.5 vs 77.1 s, all within run-to-run noise. setup time is ~0.25 s either way.

This is expected for this test: it runs the same model back-to-back, so the
transformer stays resident and is never re-read from disk — the RAM-cache size
never becomes the bottleneck. The cache bump targets a different scenario
(mixed-model workflows, where a too-small cache evicts and reloads models
between generations). It is not part of the code PR and, on this evidence,
is best treated as an optional tuning knob rather than a guaranteed speedup.

Notes

• Warm vs. cold. The first generation after a restart loads the model cold
  (setup ~2.6–4.7 s, one-time) and — because VRAM is clean — often denoises
  faster (the transformer fully loads). The first generation at a new
  resolution is slightly slower (kernel autotune for the new shape). Both first
  runs are excluded above to show steady state.
• Denoise variance. On the 8 GB card the transformer partial-loads at a
  varying % each run, so denoise swings several seconds; the averages show the
  trend. The 1280² baseline in particular varied ~82–87 s across batches, so the
  cuDNN gain is better read as ~8–12 % than a single point estimate.
• Total = setup is not included; it is denoise + VAE decode (matching the
  CSV). Text-encode is a separate node, small, and unaffected by these changes.

Related Issues / Discussions

QA Instructions

Merge Plan

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• ❗Changes to a redux slice have a corresponding migration
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

[kappacommit]

Just as a status update on this one, I'm not planning to finalize it until after we get torch bumped in the next release, I want to see if that accounts for the remaining speed differences or if there's still more to investigate

[lstein]

@Pfannkuchensack Could you test on Windows?

[Pfannkuchensack]

Findings

Medium - Test coverage: every new code path is untested, despite an existing precedent.
invokeai/backend/util/vae_working_memory.py:96 (estimate_vae_working_memory_anima)
and invokeai/app/invocations/anima_latents_to_image.py:62 (_use_tiled_decode)
are pure, deterministic functions with no test coverage, even though the repo
already unit-tests this exact category in
tests/app/invocations/test_z_image_working_memory.py. The PR adds zero tests
for the estimator, the tiling decision, the OOM fallback, or the encode
disable_tiling() call. The estimator is the riskiest part: a too-low constant
silently under-reserves and causes OOM; a regression in the
tile_size/element_size/operation branching would be invisible. Evidence
chain: behavior depends entirely on scaling_constant, the tile_size is not None
branch, and element_size scaling (lines 110-122), none of which is exercised.
To expose this issue, add a test that calls estimate_vae_working_memory_anima
with a stub vae whose parameters() yield an fp16 tensor and asserts: (a)
decode/encode use the 2900/1450 constants, (b) tile_size=512 yields the fixed
512*512*element_size*2900*1.25 value independent of image_tensor dims, and
(c) tile_size=None decode scales by LATENT_SCALE_FACTOR**2 versus encode.
To expose the threshold logic, add a test that asserts _use_tiled_decode
returns False for device.type != "cuda" and, with a patched
torch.cuda.get_device_properties, flips at the 0.7 * total_memory boundary.

Low/Medium - OOM fallback only catches torch.cuda.OutOfMemoryError, not the generic OOM path.
invokeai/app/invocations/anima_latents_to_image.py:149 catches
torch.cuda.OutOfMemoryError. The allocator's cudaMalloc failure raises that
subclass, but OOM surfaced from inside cuDNN/cuBLAS kernels (which this very PR
newly prefers via the denoise change, and which the Wan VAE convs also hit)
frequently raises a plain RuntimeError("... out of memory") that is not an
instance of OutOfMemoryError. In that case the tiling fallback at lines 150-161
never runs and the decode fails hard on exactly the small-VRAM GPUs the PR
targets. The chain: full decode under VRAM pressure -> conv kernel OOM as
RuntimeError -> except torch.cuda.OutOfMemoryError does not match ->
propagates, fallback dead. This is a robustness regression-of-intent (the
fallback is advertised as the safety net but does not cover a common OOM shape).
To expose this issue, add a test that monkeypatches vae.decode to raise a plain
RuntimeError("CUDA out of memory") on first call and asserts the invocation
still retries with tiling (or, if the narrow catch is intentional, document why
generic OOM is excluded).

Low - Behavior/output changes without an invocation version bump.
invokeai/app/invocations/anima_latents_to_image.py:49 keeps version="1.0.2"
and invokeai/app/invocations/anima_image_to_latents.py:49 keeps version="1.0.0",
yet both nodes change decode/encode behavior. Tiled decode produces
pixel-different output from a full decode (blend seams via blend_v/blend_h),
and whether tiling engages depends on total_memory and resolution. So identical
latents yield hardware-dependent, non-reproducible output, and the invocation
cache (keyed partly on node version) will not distinguish old vs new behavior.
These are Classification.Prototype nodes so the bar is lower, but a version
bump is the repo's normal signal for changed node semantics.

[kappacommit]

this PR is ready for review, based on my testing on an 8gb GPU, anima now performs within a stone's throw of Comfy's performance

[kappacommit]

Feedback addressed

[lstein]

Works as advertised.