Skip to content

Releases: ggml-org/llama.cpp

Release list

b10107

b10107 Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 24 Jul 14:01
b10107
c0bc859
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

hexagon: fix Windows crash when op_poll is enabled (#26029)

Website:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Assets 27

b10106

b10106

Choose a tag to compare

View all tags
@github-actions github-actions released this 24 Jul 11:34
b10106
1425386
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

CUDA: fix external compilation of q1_0 MMQ (#25778)

Website:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b10105

b10105

Choose a tag to compare

View all tags
@github-actions github-actions released this 24 Jul 09:34
b10105
e6dd0e2
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

args: refactor mlock/mmap/directio into load-mode (#20834)

  • args: overhaul mmap/mlock/dio into single arg

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • docs: update docs with llama-gen-docs

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • chore: satisfy code quality

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • args: make the + sign an actual modifier now

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • chore: general code clean up + comments

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • arg: fix deprecated flags support

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • arg: quick sanity check

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • bench: sync llama-bench argument parsing

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • fix: bugfix variable behaviour + llama-bench lm column size

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • arg: inverse commands should do the opposite instead of doing nothing

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • bench: fix incorrect dash

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • bench: fix missing modifiers for deprecated flags

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • llama: switch back to thread_local

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • arg: switch back to single enum

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • docs: update arg docs

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • chore: fix missing mlock from llama_load_mode_from_str + cleanup llama-bench

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • llama: fix mlock not activating

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • arg: add deprecation warning when old and new flags are combined

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • arg: cont add comment for todo in the future

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • docs: sync with upstream

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • docs: re-sync with upstream again

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

Website:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b10103

b10103

Choose a tag to compare

View all tags
@github-actions github-actions released this 24 Jul 06:14
b10103
c588c4f
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

metal : add f16 type support to leaky relu (#25981)

Website:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b10099

b10099

Choose a tag to compare

View all tags
@github-actions github-actions released this 23 Jul 23:34
b10099
1a064ab
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

CUDA: Improve NVFP4 W4A4 activation quantization (#25730)

  • Squash history before conflict-resolution during rebase on master

WIP commit

Add 32-byte loads, restore per-block amax

Use nvfp4x4 intrinsic when available

Fuse per-channel amax and quantization kernels

Do pointer arithmetic only once on x

Remove unnecessary ternary in the load

We assert on host side that ne00 is 64-aligned

Add back scale-search, but optimize it with intrinsics

Code cleanup

Make scale in MMQ-epilogue NVFP4-specific/restrictive for now

Remove unneeded include, add comment

Fix trailing whitespace

Guard builtin_align(32) struct to NVIDIA

Seems like HIP doesn't have this available, see https://github.com/ggml-org/llama.cpp/actions/runs/29438651734/job/87431623001

  • compiler massaging to avoid unnecessary LDCs

  • kvalues_mxfp4 -> kvalues_nvfp4 in quantize_mmq_nvfp4

  • Always pass in src1_scale.ptr

  • Extract ggml_cuda_is_aligned helper

Website:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b10098

b10098

Choose a tag to compare

View all tags
@github-actions github-actions released this 23 Jul 20:12
b10098
0278d83
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

hexagon: activation ops update (#25974)

  • hex-geglu: optimized all-in-one geglu microkernel

  • hex-geglu: enable non-contiguous src and strided DMA

  • hex-act: enable non-contiguous srs and strided DMA for rest of ACT ops

  • hex-act: generalize GLU per-thread functions via DEFINE_GLU_PER_THREAD macro

  • hexagon: move UNARY_SILU and UNARY_GELU to unary-ops

  • hex-act: replace the generic ops_context scratchpad usage with a local htp_vtcm_layout computation per act op.

Co-authored-by: Max Krasnyansky maxk@qti.qualcomm.com

Website:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b10094

b10094

Choose a tag to compare

View all tags
@github-actions github-actions released this 23 Jul 18:28
b10094
6d5a910
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

common: infer the speculative type from the draft repo sidecars (#25989)

With -hfd pointing to a repo that ships mtp-/dflash-/eagle3- sidecars
and no --spec-type given, the draft resolved to a full model while the
sidecar was the intended draft.

When the speculative types are still at their default, discover the
sidecars of the draft repo, pick the first available following the
existing mtp > dflash > eagle3 priority, and set the corresponding
type, so this now works without any extra flag:

llama-server -hf repo:Q3_K_M -hfd repo:Q8_0

An explicit --spec-type disables the inference, and a draft repo
without sidecars keeps resolving to a full model as before.

Website:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b10093

b10093

Choose a tag to compare

View all tags
@github-actions github-actions released this 23 Jul 17:19
b10093
f534da2
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Fix DeepSeek4 crafted template (#25414)

  • chat: fix DS4 template to explicitly follow reference behavior

  • Support DeepSeekv4 flag (drop_reasoning).

  • fix: hook DS3.2 parser for DS4 as well

  • fix: add tool result reordering

  • fix: post-merge

Website:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b10092

b10092

Choose a tag to compare

View all tags
@github-actions github-actions released this 23 Jul 05:30
b10092
3ce7da2
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

ggml: enable PowerPC backend variants on AIX (#25983)

  • ggml: enable PowerPC backend variants on AIX

Allow the PowerPC CPU backend variants to be built on AIX by extending the platform check in the CMake configuration. This reuses the existing PowerPC backend implementations without changing their behavior.

Also fix a missing semicolon in the PowerPC Q0 matmul implementation.

  • Fix missing semicolon in sgemm.cpp

Website:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b10091

b10091

Choose a tag to compare

View all tags
@github-actions github-actions released this 22 Jul 21:07
b10091
b4d6c7d
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

ci : fix SYCL package shared library lookup (#25987)

Website:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading