Skip to content

CUDA crash (silent) at ~8K+ context on Jetson Orin Nano Super (SM87) with turbo4 KV #1

Description

@SimesD

Environment

  • Hardware: NVIDIA Jetson Orin Nano Super (SM87, 8GB LPDDR5 unified memory, 7619 MiB reported by CUDA)
  • L4T: R36.4.7, kernel 5.15.148-tegra
  • CUDA: 12.6
  • Model: gemma4-e2b-IQ2_M.gguf (Gemma 4 E2B)
  • Build: cmake -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=87 -DCMAKE_BUILD_TYPE=Release
  • Branch: feature/turboquant-kv-cache (e93b7c5)

Flags

--cache-type-k turbo4 --cache-type-v f16 -ngl 99 -c 16384 --no-mmap --no-warmup -b 128 -ub 128

Behaviour

Server starts and handles short prompts correctly (<2K tokens). At ~8K+ tokens the server crashes silently — no stderr output, no log entries, process just disappears.

Short context (~43 tokens): works, 23.5 tok/s decode
Medium context (~1.5K tokens): works, 24.4 tok/s decode
Long context (~4K tokens): works, 23.5 tok/s decode
~8K tokens: silent crash

Control

Stock llama.cpp llama-server (same model, same flags minus turbo4) handles 8K+ without issue, ruling out OOM or L4T allocation bug.

Suspected cause

The silent crash with no error output suggests a hard CUDA abort in the kernel — likely the warp-cooperative 512-WHT write path or a syncthreads deadlock in the D=512 SET_ROWS path under higher token counts. The document describing this implementation mentions both a syncthreads deadlock fix (warps exiting j-loop early) and a warp reduction deadlock (only 8/32 threads participating in shfl_down). These may not be fully resolved for SM87 at longer sequences.

Notes

  • CUDA OOM is ruled out — 7619 MiB unified memory, model only uses ~1150 MiB, plenty of headroom
  • turbo3/turbo2 not yet tested (may exhibit same issue)
  • Crash occurs on fresh boot with jetson_clocks enabled

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions