API Error: terminated (cause: Body Timeout Error)

[denysobukh]

What happened?

run prompt to process web pages

What did you expect to happen?

run is processed

Client information

Client Information

/about

│ Status │
│ │
│ Qwen Code 0.16.2 (7aba696) │
│ Runtime Node.js v26.0.0 / npm 11.12.1 │
│ LSP disabled │
│ OS darwin arm64 (24.6.0) │
│ │
│ Auth API Key - openai │
│ Base URL http://localhost:1234/v1 │
│ Model /Users/denysobukhov/Models/unsloth/Qwen3.6-27B-UD-MLX-4bit │
│ Fast Model /Users/denysobukhov/Models/unsloth/Qwen3.6-27B-UD-MLX-4bit │
│ Session ID b2ad61fe-a53a-4467-8f9e-931676613ddf │
│ Sandbox no sandbox │
│ Proxy no proxy │
│ Memory Usage 207.6 MB

Login information

settings.json

{
  "env": {
    "QWEN_CUSTOM_API_KEY_OPENAI_HTTP_LOCALHOST_1234_V1": "sk-empty"
  },
  "modelProviders": {
    "openai": [
      {
        "id": "/Users/denysobukhov/Models/unsloth/Qwen3.6-27B-UD-MLX-4bit",
        "name": "Qwen3.6-27B-UD-MLX-4bit",
        "baseUrl": "http://localhost:1234/v1",
        "envKey": "QWEN_CUSTOM_API_KEY_OPENAI_HTTP_LOCALHOST_1234_V1",
        "generationConfig": {
          "timeout": 6000000,
          "modalities": {
            "image": false
          }
        }
      },
      {
        "id": "qwen3-vl-32b",
        "name": "Qwen3 VL 32B Local GGUF",
        "baseUrl": "http://127.0.0.1:1234/v1",
        "description": "Local Qwen3-VL-32B-Instruct GGUF via llama.cpp llama-server",
        "envKey": "OPENAI_API_KEY",
        "generationConfig": {
          "contextWindowSize": 65536,
          "maxOutputTokens": 4096,
          "temperature": 0.6,
          "topP": 0.95,
          "timeout": 900000,
          "maxRetries": 1
        }
      }
    ]
  },
  "security": {
    "auth": {
      "selectedType": "openai"
    }
  },
  "model": {
    "name": "/Users/denysobukhov/Models/unsloth/Qwen3.6-27B-UD-MLX-4bit"
  },
  "$version": 4,
  "permissions": {
    "allow": [
      "mcp__web-search__full-web-search",
      "WebFetch(huggingface.co)",
      "Bash(ls *)",
      "Bash(python3 *)",
      "Bash(cp *)",
      "Bash(source *)",
      "Bash(python *)",
      "Bash(pip show *)",
      "Bash(find *)",
      "Bash(du *)",
      "Bash(xargs *)",
      "Bash(awk *)",
      "Bash(readlink *)",
      "Bash(head *)",
      "Bash(tesseract *)",
      "Bash(iconv *)",
      "Bash(do)",
      "Bash(done)"
    ]
  },
  "mcpServers": {
    "memory": {
      "httpUrl": "http://127.0.0.1:8765/mcp",
      "timeout": 30000,
      "trust": true
    },
    "context7": {
      "url": "https://mcp.context7.com/mcp",
      "headers": {
        "CONTEXT7_API_KEY": "xx",
        "Accept": "application/json, text/event-stream"
      },
      "httpUrl": "https://mcp.context7.com/mcp"
    },
    "web-search": {
      "command": "node",
      "args": [
        "/Users/denysobukhov/mcp-servers/web-search-mcp-v0.3.2/dist/index.js"
      ]
    },
    "gmail": {
      "httpUrl": "https://gmailmcp.googleapis.com/mcp/v1",
      "oauth": {
        "enabled": true,
        "clientId": "200917708091-54d4pbdllrpporrh5lt8m3cuf4ulu6ml.apps.googleusercontent.com",
        "clientSecret": "xx",
        "scopes": [
          "https://www.googleapis.com/auth/gmail.readonly",
          "https://www.googleapis.com/auth/gmail.compose"
        ]
      }
    }
  },
  "mcp": {
    "excluded": [
      "context7"
    ]
  },
  "tools": {
    "approvalMode": "auto-edit"
  },
  "ui": {
    "theme": "Qwen Light"
  },
  "general": {
    "preferredEditor": "vim",
    "gitCoAuthor": {
      "commit": true
    }
  },
  "experimental": {
    "emitToolUseSummaries": true
  }
}

Anything else we need to know?

run server with:

#!/usr/bin/env bash
set -euo pipefail

VENV="${VENV:-$HOME/venvs/venv-mlx}"
MODEL="${MODEL:-$HOME/Models/unsloth/Qwen3.6-27B-UD-MLX-4bit}"
HOST="${HOST:-0.0.0.0}"
PORT="${PORT:-1234}"

if [[ ! -d "$VENV" ]]; then
  echo "ERROR: venv not found: $VENV"
  exit 1
fi

if [[ ! -d "$MODEL" ]]; then
  echo "ERROR: model directory not found: $MODEL"
  exit 1
fi

source "$VENV/bin/activate"

echo "Venv:  $VENV"
echo "Model: $MODEL"
echo "Host:  $HOST"
echo "Port:  $PORT"
echo

exec python -m mlx_lm.server \
  --model "$MODEL" \
  --host "$HOST" \
  --port "$PORT" \
  --temp 0.6 \
  --top-p 0.95 \
  --top-k 20 \
  --min-p 0.05 \
  --max-tokens 32677 \
  --prompt-cache-bytes 1G \
  --decode-concurrency 1 \
  --prompt-concurrency 1 \
  --chat-template-args '{"preserve_thinking":true}'

[qwen-code-ci-bot]

Thanks for the detailed configuration info. To help diagnose this timeout error, could you share:

1. What action triggers the timeout? What prompt or command were you running when the error occurred?
2. Debug logs: Can you run with flag and share the logs from a failing session?
3. Server health: Is the responding normally to other requests? Any errors in the server logs?
4. Timing: Does this happen immediately, after some time, or only with certain types of prompts?
5. Version check: Did this work on a previous version of Qwen Code, or is this a new setup?

The timeout you're seeing suggests the model server isn't responding within the expected window. Debug logs would help identify if this is a connection issue, a model inference timeout, or something else.

[denysobukh]

1 prompt i run

 тебе нужно составить словарь старых слов, наречий, вышедших из употреблений. на русском языке. Источник https://forum.vgd.ru/1621/82952/ и подстраницы этого раздела. словарь запиши в словарь.md . ничего не
  додумывай! не анализируй глубоко! только список : слово - определение и примеры если есть

3
server. side logs

2026-05-28 16:37:08,746 - WARNING - Failed to parse tool call (ValueError: No function provided.) — tool text was likely truncated mid-generation.
----------------------------------------
Exception occurred during processing of request from ('127.0.0.1', 58625)
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.14/3.14.5/Frameworks/Python.framework/Versions/3.14/lib/python3.14/socketserver.py", line 697, in process_request_thread
    self.finish_request(request, client_address)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.14/3.14.5/Frameworks/Python.framework/Versions/3.14/lib/python3.14/socketserver.py", line 362, in finish_request
    self.RequestHandlerClass(request, client_address, self)
    ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/denysobukhov/venvs/venv-mlx/lib/python3.14/site-packages/mlx_lm/server.py", line 1716, in <lambda>
    lambda *args, **kwargs: handler_class(
                            ~~~~~~~~~~~~~^
        response_generator,
        ^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        **kwargs,
        ^^^^^^^^^
    ),
    ^
  File "/Users/denysobukhov/venvs/venv-mlx/lib/python3.14/site-packages/mlx_lm/server.py", line 1073, in __init__
    super().__init__(*args, **kwargs)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.14/3.14.5/Frameworks/Python.framework/Versions/3.14/lib/python3.14/socketserver.py", line 766, in __init__
    self.handle()
    ~~~~~~~~~~~^^
  File "/opt/homebrew/Cellar/python@3.14/3.14.5/Frameworks/Python.framework/Versions/3.14/lib/python3.14/http/server.py", line 496, in handle
    self.handle_one_request()
    ~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/opt/homebrew/Cellar/python@3.14/3.14.5/Frameworks/Python.framework/Versions/3.14/lib/python3.14/http/server.py", line 484, in handle_one_request
    method()
    ~~~~~~^^
  File "/Users/denysobukhov/venvs/venv-mlx/lib/python3.14/site-packages/mlx_lm/server.py", line 1202, in do_POST
    self.handle_completion(request, stop_words)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/Users/denysobukhov/venvs/venv-mlx/lib/python3.14/site-packages/mlx_lm/server.py", line 1514, in handle_completion
    self.wfile.write(f"data: {json.dumps(resp)}\n\n".encode())
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.14/3.14.5/Frameworks/Python.framework/Versions/3.14/lib/python3.14/socketserver.py", line 845, in write
    self._sock.sendall(b)
    ~~~~~~~~~~~~~~~~~~^^^
BrokenPipeError: [Errno 32] Broken pipe
----------------------------------------

[denysobukh]

client debug log

b375e767-7baa-42b4-b592-a980ae7f9238.txt

[doudouOUC]

Root Cause Identified

Thanks for the server logs — they pinpointed the issue. The BrokenPipeError is a symptom of the real problem.

What's happening

The timeout comes from undici's bodyTimeout — an idle timeout between SSE chunks during streaming:

1. Request sent → server responds with HTTP 200 instantly ✓
2. Streaming begins, first chunk arrives ✓
3. Model pauses to generate the next token (complex tool call JSON)
4. If no data arrives for 300 seconds, undici kills the connection → Body Timeout Error
5. Server tries to write to the closed socket → BrokenPipeError: [Errno 32] Broken pipe

Your server log confirms this: "Failed to parse tool call — tool text was likely truncated mid-generation" — the generation was still in progress but the client disconnected before it completed.

Why your timeout: 6000000 setting doesn't help

The timeout in generationConfig only controls the initial HTTP response wait (time between sending the request and receiving the first HTTP headers). Once the server responds with 200 OK and streaming starts, that timer is cleared.

The body-level idle timeout (300s between any two chunks) comes from Node.js's built-in undici HTTP client and is not exposed as a user-configurable option in Qwen Code today.

Why this path is affected

In the codebase, there are 3 fetch paths with different timeout behavior:

Runtime	Proxy configured?	bodyTimeout	Status
Bun	N/A	disabled (timeout: false)	✓ Works
Node.js	Yes	disabled (bodyTimeout: 0)	✓ Works
Node.js	No	300s default (undici built-in)	✗ Breaks with slow models

Most local model users (connecting to localhost) don't configure a proxy, so they hit the third path.

Fix

We've submitted a fix in #4667 that adds a configurable generationConfig.bodyTimeout field (default 0 = disabled) and creates a custom undici dispatcher for the no-proxy Node.js path. This disables the 300s body timeout, consistent with the proxy and Bun paths.

[joleuger]

I have faced the same issue. On llama.cpp-side, I have created a PR to send SSE keepalive messages ggml-org/llama.cpp#23994 This might be beneficial in situations where the connection really hangs.