Skip to content

Releases: xorbitsai/inference

Release list

v3.1.0.rc1

v3.1.0.rc1 Pre-release
Pre-release

Choose a tag to compare

View all tags
@XprobeBot XprobeBot released this 29 Jul 07:44
v3.1.0.rc1
a402bb2
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's new in 3.1.0.rc1 (2026-07-29)

These are the changes in inference v3.1.0.rc1.

New features

  • feat(ui): add SSO/OIDC login button to login page by @m199369309 in #5218
  • feat: launch model worker multi select by @maoyuehui in #5222
  • feat(audio): batch Qwen3-ASR transcriptions by @qinxuye in #5172
  • feat(webui): add auto-refresh to audit center by @m199369309 in #5226
  • feat(webui): show API key name and model ID columns in audit center by @m199369309 in #5229
  • feat(webui): multi-dashboard monitor center with admin config dialog by @m199369309 in #5225
  • feat(frontend): auto-reload shell routes on export directory change by @m199369309 in #5243
  • feat(vllm): populate logprobs/top_logprobs in completion responses by @SuperMarioYL in #5245
  • feat(webui): support multi-node selection in log center by @m199369309 in #5246
  • feat: speculative decoding with paired drafter checkpoints by @qinxuye in #5241
  • feat(engine): [image, audio] sglang image vllm asr engines by @OliverBryant in #5121
  • feat(webui): add missing permission scopes (models:register, logs:list, monitor:view) to UI by @m199369309 in #5254

Enhancements

Bug fixes

  • fix(embedding/rerank): fix sentence_transformers load failures from torch-ecosystem package mismatches by @OliverBryant in #5210
  • fix(webui): show model engine on running-model detail page by @OliverBryant in #5211
  • fix(pypiserver): emit model source paths with forward slashes by @OliverBryant in #5216
  • fix: (UI) Update the label of n_gpu except for LLM and Image types by @maoyuehui in #5217
  • FIX: mitigate Slow HTTP DoS (Slowloris) by enforcing a request receipt deadline by @OliverBryant in #5213
  • fix(api): cast replica to int to prevent TypeError on multi-replica launch by @m199369309 in #5220
  • fix(venv): pin torch to system version to prevent torchvision ABI crash on relaunch by @OliverBryant in #5212
  • BUG: worker keeps stale model state after a launch fails during async engine load by @OliverBryant in #5193
  • fix(audio/indextts2): declare the missing IndexTTS2 virtualenv dependencies (#5201) by @OliverBryant in #5223
  • fix(vllm): add #system_numpy# and pin vllm to 0.21.0 for Ornith/qwen3.5/qwen3.6 by @m199369309 in #5235
  • fix(mlx): keep VLM generation on a dedicated thread by @qinxuye in #5234
  • fix: disambiguate replica model UIDs with -rep{n} suffix by @Solaris-star in #5215
  • fix: allow virtualenv packages to override system markers by @LauraGPT in #5204
  • fix(embedding): re-normalize per-item index after batch split by @m199369309 in #5240
  • FIX: suppress GPU warnings in CPU image by @leslie2046 in #5238
  • fix(i18n): change SSO button text from Keycloak to generic SSO by @m199369309 in #5244
  • bug: cover list_models tolerance for stale replica uids by @OliverBryant in #5168
  • fix: stabilize Jina v5 and Qwen3 reranker deployment by @OliverBryant in #5232
  • FIX: worker reconnection after supervisor restart by @leslie2046 in #5250
  • fix(webui): improve worker node display in multi-select by @m199369309 in #5255
  • fix(worker): skip redundant virtualenv setup when replicas share the same venv by @m199369309 in #5256

Documentation

Others

  • ci: use merge-base diff for PR GPU tests by @qinxuye in #5224
  • docs: update release notes links from xinference.io to xinference.co by @martinma51 in #5248
  • Consolidate packaging configuration into pyproject.toml by @OliverBryant in #5249

New Contributors

Full Changelog: v3.0.0...v3.1.0.rc1

Contributors

qinxuye, leslie2046, and 8 other contributors
Loading

v3.0.0

v3.0.0 Latest
Latest

Choose a tag to compare

View all tags
@XprobeBot XprobeBot released this 19 Jul 03:10
v3.0.0
8ca6a62
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's new in 3.0.0 (2026-07-19)

These are the changes in inference v3.0.0.

New features

  • FEAT: Switch default frontend to Next.js app by @OliverBryant in #5111
  • feat: (UI) API Key, User Management, refreshToken, change password after first login by @maoyuehui in #5143
  • feat(api): validate system message ordering for strict Qwen3-family chat templates by @m199369309 in #5149
  • feat(embedding): support BCE embeddings with vLLM by @amumu96 in #5158
  • Feat: (UI) Added Security Settings Page、Audit Center Page by @maoyuehui in #5153
  • feat: enable advanced database-backed authentication by default by @OliverBryant in #5144
  • FEAT: production-ready Docker Compose deployment with offline private PyPI profile by @OliverBryant in #5174
  • feat(auth): consolidate to database-backed auth and add offline password reset by @OliverBryant in #5180
  • FEAT(webui): show per-model GPU memory in Running Models by @OliverBryant in #5186
  • feat(llama.cpp): install GPU xllamacpp wheel from CUDA-matched index by @OliverBryant in #5185
  • FEAT: prebuilt pypiserver mirror image as the offline compose package source by @qinrui777 in #5048
  • FEAT: slim GPU Docker image without pre-installed inference engines by @OliverBryant in #5188

Enhancements

Bug fixes

  • fix(auth): advanced-auth live-read via DB-current permissions by @m199369309 in #5135
  • fix(vllm): flashinfer AOT post-install hook for sm_120 Blackwell by @m199369309 in #5137
  • fix(audio): bump funasr pin for Fun-ASR-Nano models to ~=1.3 by @m199369309 in #5140
  • fix(frontend): route params via usePathname(); remove Gradio, add LaTeX rendering by @OliverBryant in #5146
  • fix: replace Korean text with Japanese in ja locale launchModel keys by @kejhz653-stack in #5154
  • fix(embedding): implement truncate_prompt_tokens, OOM eviction, and wait_for_load after recovery by @m199369309 in #5151
  • fix(tool_parser): fix deepsee-v4 stream tool calls error by @llyycchhee in #5155
  • fix(worker): use os.replace for atomic launch_args persist on Windows (#5161) by @Anai-Guo in #5162
  • fix(supervisor): resolve comma-separated worker_ip in sharded launch (#5138) by @Anai-Guo in #5160
  • fix(llama.cpp): resolve mmproj against the model directory so image input works by @Anai-Guo in #5170
  • fix(oauth2): revoke refresh tokens on password reset by @OliverBryant in #5177
  • Fix: (UI) copyToClipboard 、fix the locales Running Model Page by @maoyuehui in #5175
  • fix(core): free GPU memory when stopping pytorch rerank models by @OliverBryant in #5173
  • BUG: sglang venv inherits incompatible parent transformers under skip_installed by @OliverBryant in #5181
  • fix: honor virtualenv exemption in engine matching for multimodal LLM, embedding and rerank by @OliverBryant in #5179
  • fix(image/ocr): import torchvision lazily in deepseek_ocr (#5189) by @Anai-Guo in #5191
  • fix(worker): create model subpool after virtualenv is installed by @kejhz653-stack in #5196
  • fix: pass torch version to offline package builds by @qinxuye in #5200
  • fix: mirror dependencies declared by wheels built from sdists by @OliverBryant in #5202

Documentation

Others

  • ci: narrow GPU test group detection to paths that actually affect GPU tests by @OliverBryant in #5148
  • refactor: move XINFERENCE_MODEL_ACTOR_AUTO_RECOVER_LIMIT to constants.py by @m199369309 in #5152
  • ci: fix Metal CI failures with latest MLX on macOS 26 runners by @OliverBryant in #5125
  • docs(monitor): enhance metrics documentation with cluster label, status detail, and HTTP middleware by @m199369309 in #5184
  • chore: remove legacy React web UI now that Next.js frontend is the default by @OliverBryant in #5145

Full Changelog: v2.12.0...v3.0.0

Contributors

qinxuye, m199369309, and 9 other contributors
Assets 2
Loading

v2.12.0

v2.12.0

Choose a tag to compare

View all tags
@XprobeBot XprobeBot released this 04 Jul 11:45
v2.12.0
0a310a5
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's new in 2.12.0 (2026-07-04)

These are the changes in inference v2.12.0.

New features

  • feat: gate model remote-code execution behind a config switch by @qxyuan853 in #5027
  • FEAT: [UI] add API key management and user management pages by @yiboyasss in #5065
  • FEAT: [UI] ability panel add(Embed、Rerank) by @maoyuehui in #5078
  • feat(monitor): split Grafana dashboard into 6 sub-dashboards with config store by @m199369309 in #5073
  • feat(metrics): unify worker labels with supervisor and add replica_index by @m199369309 in #5075
  • feat: register VibeThinker 1.5B / 3B (transformers + vLLM) by @xiaoyesoso in #5085
  • feat: register Nex-N2 series (mini / Pro / Pro-fp8) by @xiaoyesoso in #5094
  • feat(log): add daily+size rotation mode with multi-process safety by @m199369309 in #5083
  • feat(models): add model autostart config by @bluefish-08 in #5076
  • FEAT(webui): replace worker IP input with a filtered multi-select selector by @leslie2046 in #5099
  • feat: add Unlimited-OCR support by @xiaoyesoso in #5103
  • Feat: (UI) running model add Try to API and add useMenuAuth hook by @maoyuehui in #5098
  • feat(llm): register Ornith-1.0-35B by @m199369309 in #5119
  • feat: permission/scope alignment (core — no live-read) by @m199369309 in #5133

Enhancements

Bug fixes

  • fix: avoid blocking the event loop in async update_model_type by @kratos0718 in #5055
  • BUG: Fix llama.cpp backend by @codingl2k1 in #5056
  • FIX: [model] chat template analysis error by @llyycchhee in #5060
  • fix(embedding): pin peft/transformers for jina-embeddings-v5 series by @m199369309 in #5061
  • fix(docker): cap fastapi<0.137 in image requirements to match setup.cfg (#5063) by @Anai-Guo in #5072
  • fix(device_utils): tolerate unsupported nvmlDeviceGetUtilizationRates in WSL2 by @Anai-Guo in #5070
  • fix(audio): resolve Qwen3-ASR forced aligner via ModelScope when it is the active source by @Anai-Guo in #5077
  • fix(core): prevent worker metrics loss when cache_tracker init fails by @m199369309 in #5071
  • fix(monitor): add model_name label to api_key_requests_total counter by @m199369309 in #5074
  • fix: drop stale running models without replica info in supervisor.list_models by @xiaoyesoso in #5087
  • fix(core): harden sub-pool creation and clean GPU orphans on startup by @m199369309 in #5095
  • fix: handle DynamicCache in get_batch_size_and_seq_len_from_kv_cache when HybridCache is importable by @xiaoyesoso in #5086
  • fix(oauth2): verify JWT expiration in AuthService (#5058) by @qfmy83 in #5088
  • fix(oauth2): block privilege escalation in user permission grants (#5058) by @qfmy83 in #5089
  • fix(llm/cache): clear error and ensure cache parent dir on file:// model_uri (#5091) by @Anai-Guo in #5097
  • fix(vllm): silence repeated GuidedDecodingParams info log per request by @m199369309 in #5120
  • fix(llm): install ninja in qwen3.5/qwen3.6 vLLM venv for FlashInfer JIT by @m199369309 in #5118
  • fix(llm): normalize tool_call arguments to dict for Qwen3 family templates by @m199369309 in #5129

Documentation

Others

New Contributors

Full Changelog: v2.11.0...v2.12.0

Contributors

qinxuye, leslie2046, and 14 other contributors
Loading

v2.11.0

v2.11.0

Choose a tag to compare

View all tags
@XprobeBot XprobeBot released this 19 Jun 10:54
v2.11.0
7a6807c
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's new in 2.11.0 (2026-06-19)

These are the changes in inference v2.11.0.

New features

Enhancements

Bug fixes

  • fix(supervisor): self-heal worker registry after supervisor restart by @m199369309 in #4998
  • fix(metrics): drop stale series and add unexpected termination gauge by @m199369309 in #4999
  • fix(worker): ensure subpool monitor starts after append_sub_pool by @m199369309 in #5004
  • fix(model_registration):add flexible type by @llyycchhee in #5007
  • fix(model): use os._exit for OOM to trigger pool recovery by @m199369309 in #5005
  • fix(venv): add sentence_transformers dependence by @llyycchhee in #5009
  • fix(supervisor): evict dead replica from round-robin when auto-recover exhausted by @m199369309 in #5006
  • fix(worker): pop launch_ts in recover_model to avoid strict-constructor crashes by @m199369309 in #5012
  • fix(deploy): fix dockerfile.cpu kernels version by @llyycchhee in #5013
  • fix(logs): fix empty node filter dropdown and redesign toolbar layout by @m199369309 in #5015
  • fix(monitor): correct alert rule labels/expressions and add 4 new rules by @m199369309 in #5016
  • fix: pin FastAPI below 0.137 for metrics middleware by @qinxuye in #5042
  • fix(core): sync model_serve_count gauge on stream request completion by @m199369309 in #5041
  • fix(vllm): force spawn for single-GPU EngineCore and fix health check scheduling by @m199369309 in #5030
  • fix(worker): strip test envs before recover and clean up GPU orphans by @m199369309 in #5031
  • fix(rerank/llama.cpp): honor flat launch kwargs (n_ctx/n_batch/n_ubatch) so rerankers aren't capped at default ubatch (#5001) by @Anai-Guo in #5046
  • fix(anthropic): honor top-level system prompt and inline system messages (#5037) by @Anai-Guo in #5049
  • fix(vllm/patches): auto-apply hybrid KV cache patch to Qwen3-Next by @Anai-Guo in #5052
  • fix(audio): warn and ignore non-zero temperature for Qwen3-ASR instead of raising by @Anai-Guo in #5054

Documentation

Others

New Contributors

Full Changelog: v2.10.0...v2.11.0

Contributors

qinxuye, xiaoyesoso, and 7 other contributors
Loading

v2.10.0

v2.10.0

Choose a tag to compare

View all tags
@XprobeBot XprobeBot released this 05 Jun 23:00
v2.10.0
0701b70
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's new in 2.10.0 (2026-06-05)

These are the changes in inference v2.10.0.

New features

  • feat(logging): enhance logging system with JSON format and stdout redirect by @m199369309 in #4947
  • feat(auth): add OIDC/Keycloak SSO authentication support by @m199369309 in #4948
  • feat(audit): add comprehensive audit logging system by @m199369309 in #4951
  • feat(security): add IP/Key ban and rate limiting for brute-force protection by @m199369309 in #4949
  • feat(apikey): add description field and ban status display by @m199369309 in #4952
  • feat(monitor): add security audit panels and filebeat configurations by @m199369309 in #4953
  • feat(ui): menu reorganization, fetchWrapper auth, and i18n updates by @m199369309 in #4954
  • feat(monitor): add per-model GPU memory usage metrics by @m199369309 in #4965
  • feat(monitor): update Grafana dashboards with GPU memory panels by @m199369309 in #4969
  • feat: persist launch model configuration history server-side by @m199369309 in #4972
  • FEAT: [UI] update sidebar, login logo and favicon by @yiboyasss in #4978
  • feat: new ui (register json view, formInstance, launch model list, cache/env …) by @maoyuehui in #4966
  • feat(logging): add three-level download progress logging by @m199369309 in #4989
  • feat(ui): allow editing API key name and description in edit dialog by @m199369309 in #4991

Enhancements

Bug fixes

  • fix(vllm): set quantization="fp8" when model_format is fp8 by @m199369309 in #4959
  • fix(auth): return specific error messages for expired/disabled API keys by @m199369309 in #4963
  • fix(monitor): periodic refresh for security gauges and ban remaining API by @m199369309 in #4964
  • BUG: fix jina-embeddings-v2-base-zh deployment dependencies by @m199369309 in #4970
  • fix(monitor): capture vLLM/SGLang GPU workers via deferred PID tattoo by @m199369309 in #4977
  • fix(vllm): remove best_of for v0.21.0 by @llyycchhee in #4979
  • bug: Adapt vLLM LoRA request path parameter by @amumu96 in #4980
  • fix(logging): strip all CSI escapes and route flush() through sampling by @m199369309 in #4983
  • fix(vllm): read json_schema from schema_ so guided decoding applies by @m199369309 in #4985
  • bug: fix llama.cpp streaming tool call edge cases by @qinxuye in #4988
  • fix: Fix GPU info probe on GB10 / DGX Spark (NVML v2 memory-info fallback) by @tbraun96 in #4990
  • fix(worker): limit concurrent model launches with semaphore to prevent heartbeat timeouts by @m199369309 in #4992
  • fix(venv): allow user override by @llyycchhee in #4993
  • fix(venv): evaluate CUDA version markers dynamically by @llyycchhee in #4958

Documentation

Others

  • chattts: set weights_only=True for torch.load speaker embedding by @tonghuaroot in #4956

New Contributors

Full Changelog: v2.9.0...v2.10.0

Contributors

qinxuye, tonghuaroot, and 7 other contributors
Loading

v2.9.0

v2.9.0

Choose a tag to compare

View all tags
@XprobeBot XprobeBot released this 23 May 10:04
v2.9.0
08abd54
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's new in 2.9.0 (2026-05-23)

These are the changes in inference v2.9.0.

New features

Enhancements

Bug fixes

  • fix(ocr): replace eval() with ast.literal_eval in deepseek_ocr coordinate parsing by @Ricardo-M-L in #4872
  • fix: harden docs workflow against branch injection by @qinxuye in #4910
  • fix(ci): harden gen docs workflow by @rogercloud in #4913
  • fix: Fix tool_call messages handling in utils.py by @la1ty in #4915
  • fix(ci): update vllm to 0.21.0 for Dockerfile by @zwt-1234 in #4934

Documentation

New Contributors

Full Changelog: v2.8.0...v2.9.0

Contributors

qinxuye, rogercloud, and 7 other contributors
Loading

v2.8.0

v2.8.0

Choose a tag to compare

View all tags
@XprobeBot XprobeBot released this 09 May 11:14
v2.8.0
8391e99
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's new in 2.8.0 (2026-05-09)

These are the changes in inference v2.8.0.

New features

  • feat: TCP keepalive, reverse-channel probe, list_models cache, .pth atomic write, launch exemption by @m199369309 in #4845
  • feat: comprehensive Prometheus metrics for cluster, worker, and model observability by @m199369309 in #4857
  • feat: graceful OOM handling, worker model auto-recovery, and heartbeat log enhancement by @m199369309 in #4859
  • feat: integrate Grafana Dashboard monitoring into Web UI with Prometheus metrics enhancements by @m199369309 in #4868
  • feat: add modular vLLM post-install patch framework with hybrid KV cache fix by @m199369309 in #4879
  • feat: add Prometheus alert rules for cluster monitoring by @m199369309 in #4891
  • feat: move dashboard to monitor/ and update Grafana panels by @m199369309 in #4892

Enhancements

Bug fixes

  • fix: resolve Mixed Content blocking for Launch Web UI behind HTTPS proxy by @m199369309 in #4856
  • fix(vllm): support for v0.19.0 by @llyycchhee in #4862
  • fix: resolve host/model-venv transformers version conflict for embedding/rerank engines by @m199369309 in #4864
  • fix: correct PromQL operator precedence in Grafana dashboard panels by @m199369309 in #4873
  • fix: Fix vLLM /v1/completions TypeError in Qwen3.5 by @la1ty in #4874
  • fix: improve Grafana dashboard panel layout, PromQL expressions, and i18n by @m199369309 in #4876
  • fix: Qwen3.5 tool, ERROR Can't parse single qwen tool call output by @bleakie in #4870
  • fix: differentiate jina-embeddings v3/v4 task parameter handling by @m199369309 in #4881
  • fix: Set default enable-thinking option in command line to False by @la1ty in #4875
  • fix: upgrade TTFT metric to Histogram with stream label and add non-stream TTFT recording by @m199369309 in #4890

Documentation

New Contributors

Full Changelog: v2.7.0...v2.8.0

Contributors

qinxuye, m199369309, and 6 other contributors
Loading

v2.7.0

v2.7.0

Choose a tag to compare

View all tags
@XprobeBot XprobeBot released this 25 Apr 10:59
v2.7.0
df4c7cb
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's new in 2.7.0 (2026-04-25)

These are the changes in inference v2.7.0.

New features

Enhancements

Bug fixes

  • fix: replace eval() with safe alternatives to prevent RCE in tool parsers by @Ricardo-M-L in #4786
  • fix: support JSON object parameters in CLI by @Ricardo-M-L in #4787
  • fix: support Jina API task parameters for jina-embeddings-v4 by @Ricardo-M-L in #4788
  • fix(ui): handle mixed dict and ChatMessage types in history by @qinxuye in #4814
  • fix(vllm): fix gemma-4 tool calls by @llyycchhee in #4815
  • fix(docker): unpin torchcodec to fix 503 error on reranker/embedding model load by @FlintyLemming in #4817
  • fix: handle missing 'cpu' key in get_cluster_device_info to prevent KeyError 500 by @m199369309 in #4822
  • fix: venv concurrent creation race, cold-start lock dir, and jina-embeddings-v4 torch mismatch by @m199369309 in #4823
  • fix: dynamic CUDA version check for extra_index_url by @Gmgge in #4820
  • fix: vLLM multi-node distributed init and pipeline parallel inference by @amumu96 in #4834
  • fix: venv torchvision alignment, supervisor RPC timeouts, get_model flood protection, replica pre-check, and safe log handler by @m199369309 in #4839
  • fix: remove last message role restriction in chat completion endpoint by @amumu96 in #4833
  • fix(security): prevent pwn-request vulnerability in gen_docs workflow by @qinxuye in #4850

Documentation

Others

  • refactor(device_utils): replace if/elif chains with DeviceSpec registry by @amumu96 in #4846

New Contributors

Full Changelog: v2.5.0...v2.7.0

Contributors

qinxuye, leslie2046, and 8 other contributors
Loading

v2.5.0

v2.5.0

Choose a tag to compare

View all tags
@XprobeBot XprobeBot released this 13 Apr 04:30
v2.5.0
c9b597e
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's new in 2.5.0 (2026-04-13)

These are the changes in inference v2.5.0.

New features

Enhancements

  • ENH: update model "DeepSeek-OCR" JSON by @amumu96 in #4751
  • ENH: update 2 models JSON ("Ernie4.5", "qwen3.5") by @XprobeBot in #4754
  • ENH: update model "DeepSeek-V3.2" JSON by @amumu96 in #4762
  • ENH: update 2 models JSON ("Qwen3-ASR-0.6B", "Qwen3-ASR-1.7B") by @qinxuye in #4765
  • ENH: auto-detect PyTorch CUDA version for virtual environment setup by @qinxuye in #4766
  • ENH: update model "jina-embeddings-v4" JSON by @qinxuye in #4775
  • ENH: Optimize worker details for deployment progress tooltip. by @leslie2046 in #4746
  • ENH: update model "qwen3.5" JSON by @llyycchhee in #4782
  • ENH: update 2 models JSON ("Kokoro-82M-v1.1-zh", "Kokoro-82M") by @qinxuye in #4795
  • ENH: update model "gemma-3-it" JSON by @qinxuye in #4794
  • ENH: update models JSON [llm] by @XprobeBot in #4796
  • ENH: add lightweight heartbeat mechanism for worker liveness detection by @qinxuye in #4785
  • ENH: update model "ChatTTS" JSON by @qinxuye in #4793
  • bld: Fix the front-end UI access issue for aarch64 image by @zwt-1234 in #4743
  • bld: Fix the front-end UI access issue for aarch64 image by @zwt-1234 in #4749
  • bld: Fix the front-end UI access issue by @zwt-1234 in #4758
  • BLD: limit gptqmodel installation to specified version by @zwt-1234 in #4798

Bug fixes

Documentation

Others

New Contributors

Full Changelog: v2.4.0...v2.5.0

Contributors

qinxuye, leslie2046, and 8 other contributors
Loading

v2.4.0

v2.4.0

Choose a tag to compare

View all tags
@XprobeBot XprobeBot released this 29 Mar 03:21
v2.4.0
d189e2a
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's new in 2.4.0 (2026-03-29)

These are the changes in inference v2.4.0.

New features

Enhancements

Bug fixes

  • BUG: Fix async client FormData handling and response lifecycle issues by @qinxuye in #4687
  • BUG: MLX backend accumulates intermediate generation steps into final output (tested on 1.17.0, 2.0.0, 2.1.0) #4615 by @nasircsms in #4617
  • fix(worker): inject parent site-packages into child venv via .pth file by @nasircsms in #4692
  • BUG: launch multi gpu qwen3.5 error by @llyycchhee in #4700
  • fix(tool_call): add qwen3.5 by @llyycchhee in #4703
  • fix(qwen3.5): support tool calls by @llyycchhee in #4709
  • FIX: qwen3.5 reasoning parse by @llyycchhee in #4719
  • fix(qwen3.5): support XML-like tool call format in non-streaming mode by @amumu96 in #4715
  • FIX: webui crash when gpu_utilization is none by @leslie2046 in #4728

Documentation

New Contributors

Full Changelog: v2.3.0...v2.4.0

Contributors

qinxuye, leslie2046, and 8 other contributors
Loading