[Bug] PaddleOCR-VL: model loads but produces zero output on Windows AMD GPU

[huomutou]

Title: [Bug] PaddleOCR-VL: model loads but produces zero output on Windows AMD GPU (ROCm + Vulkan)

Environment

Item	Detail
OS	Windows 11 24H2
CPU	AMD MAX 395 (Strix Halo), 16C/32T Zen 5
GPU	AMD Radeon 8060S (gfx1151), ROCm 6.4
RAM	64 GB unified memory
llama.cpp	b8984 (latest, 2026-04-30)
Backends tested	ROCm (HIP) and Vulkan

Model

• Text model: PaddleOCR-VL-1.5.gguf (official, 893 MB)
• Vision projector: PaddleOCR-VL-1.5-mmproj.gguf (official, 841 MB)
• Also tested: community GGUF from megemini/PaddleOCR-VL-GGUF

Commands Tried

1. llama-server (ROCm / Vulkan) — hangs at slot init

llama-server.exe \
  -m PaddleOCR-VL-1.5.gguf \
  --mmproj PaddleOCR-VL-1.5-mmproj.gguf \
  --port 18115 --host 127.0.0.1 \
  --ctx-size 4096 -ngl 99 --temp 0 --no-warmup

2. llama-cli (Vulkan) — zero output

llama-cli.exe \
  -m PaddleOCR-VL-1.5.gguf \
  --mmproj PaddleOCR-VL-1.5-mmproj.gguf \
  --image test.png \
  -p "<__media__>Table Recognition:" \
  --ctx-size 4096 -ngl 99 --temp 0 --no-warmup \
  --jinja --chat-template-file chat_template_llama.jinja \
  -n 600

3. llama-mtmd-cli (ROCm / Vulkan) — zero output

4. llama-llava-cli (Vulkan) — zero output

5. llama-cli text-only (no image, no mmproj) — zero output

Expected Behavior

Model should perform OCR on input images and output recognized text.

Actual Behavior

Model and mmproj load successfully in all cases:

• Architecture paddleocr recognized ✅
• 19/19 layers offloaded to GPU ✅
• Vision encoder loaded: image_size: 1680 ✅
• KV cache allocated ✅

But inference produces zero tokens. No error, no crash, no output.

With llama-server, the server stops at:

srv    load_model: initializing slots, n_slots = 4

and never reaches the HTTP server listening message. Process exits silently.

GPU memory breakdown (no issues):

ROCm0: 89976 MiB total, 89669 MiB free
Model: 688 MiB, Context: 72 MiB, Compute: 212 MiB

Notes

• The same models were tested via Ollama 0.22.0 — Ollama rejects paddleocr architecture with unknown model architecture, so the Ollama route is also blocked.
• Related PR: model: Add PaddleOCR-VL model support #18825 (merged Feb 2026)
• Related Ollama issue: PaddleOCR-VL ollama/ollama#12685
• This appears to be a Windows + AMD GPU specific issue with the paddleocr architecture integration.

Full Logs

Model loading (no errors):

load_backend: loaded Vulkan backend
load_backend: loaded CPU backend from ggml-cpu-zen4.dll
llama_model_loader: loaded meta data with 41 key-value pairs and 165 tensors
print_info: arch = paddleocr
print_info: model type = 0.3B
print_info: model params = 466.65 M
load_tensors: offloaded 19/19 layers to GPU
clip_model_loader: has vision encoder
clip_ctx: CLIP using Vulkan0 backend
load_hparams: image_size: 1680
load_hparams: image_max_pixels: 2822400
srv load_model: loaded multimodal model
srv load_model: initializing slots, n_slots = 4
# ← hangs here, never continues

[ThePixelDiffusionPirate]

I can confirm that PaddleOCR-VL-1.5 works on Windows + AMD GPU + llama.cpp b8986.

My setup:

OS: Windows 11 24H2
GPU: AMD RX 5700 XT (Navi 10, gfx1010, RDNA 1)
Backend: Vulkan only (no ROCm/HIP installed)
llama.cpp: b8986 (2026-04-30 nightly, built with Clang 19.1.5)
Model: Official PaddleOCR-VL-1.5.gguf (935 MB) + PaddleOCR-VL-1.5-mmproj.gguf (881 MB) from Hugging Face
How I run inference: I use llama-server via the OpenAI-compatible REST API (/v1/chat/completions). No --jinja, no --chat-template-file, no external template.

The client sends a standard JSON request:

{
"model": "paddleocr-vl",
"messages": [
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": { "url": "data:image/jpeg;base64,..." }
},
{ "type": "text", "text": "OCR:" }
]
}
],
"temperature": 0,
"top_k": 1,
"max_tokens": 3800
}

Working prompts tested: OCR:, OCR markdown:, Table Recognition:, Formula Recognition:

My llama-server invocation:
llama-server.exe -m PaddleOCR-VL-1.5.gguf --mmproj PaddleOCR-VL-1.5-mmproj.gguf --port 18285 --host 127.0.0.1 --temp 0 --top-k 1 -ngl 99 --ctx-size 8192 --no-warmup

The server starts reliably, /health responds, and inference returns OCR text without issues. ---- maybe it helps 4 debug

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.