Skip to content

Support EuroBERT architecture - #19826

Merged
CISC merged 9 commits into
ggml-org:masterfrom
jina-ai:feat-jina-v5-text
Feb 26, 2026
Merged

Support EuroBERT architecture#19826
CISC merged 9 commits into
ggml-org:masterfrom
jina-ai:feat-jina-v5-text

Conversation

@maximilianwerk

@maximilianwerk maximilianwerk commented Feb 23, 2026

Copy link
Copy Markdown
Contributor

DISCLAIMER 1: This PR is created with heavy help from claude-4.5-opus-high via Cursor.

The reason I still open it:

  1. It mostly adds a new architecture, which is straight-forward and only adds a new case to the required functions.
  2. The change to imatrix.cpp is actually quite important for last-token pooling embedding models. Without this change, their imatrix calculation does not respect the actual token processing, which creates the embeddings.
  3. It is needed for the latest jina-embeddings-v5-text-nano, which ranks on place 11 on MTEB, while only having 239M parameters.

DISCLAIMER 2: I work in the Jina team.

This PR includes:

  • EuroBERT model implementation for GGUF conversion
  • C++ backend support for EuroBERT architecture
  • Change to the imatrix calculation logic for models that use last token pooling.

We converted the 4 jina-embedding-v5-text-nano models and validated their quality after conversion. For this we run several MTEB benchmark tasks and only observed a very small degradation, which is typical for gguf converted models.

In case that is relevant, here are the resulting artifacts. We merged the adapter into the base model and then ran the gguf conversion.

https://huggingface.co/jinaai/jina-embeddings-v5-text-nano-retrieval-GGUF
https://huggingface.co/jinaai/jina-embeddings-v5-text-nano-text-matching-GGUF
https://huggingface.co/jinaai/jina-embeddings-v5-text-nano-classification-GGUF
https://huggingface.co/jinaai/jina-embeddings-v5-text-nano-clustering-GGUF

@github-actions github-actions Bot added model Model specific examples python python script changes labels Feb 23, 2026
CISC
CISC reviewed Feb 23, 2026
View reviewed changes
Comment thread convert_hf_to_gguf.py Outdated
Comment thread convert_hf_to_gguf.py Outdated
Comment thread convert_hf_to_gguf.py Outdated
Comment thread convert_hf_to_gguf.py Outdated
CISC
CISC reviewed Feb 23, 2026
View reviewed changes
Comment thread src/CMakeLists.txt Outdated
maximilianwerk and others added 3 commits February 23, 2026 15:22
@maximilianwerk
WIP: Add EuroBERT support with autoformatting changes
73b7608 
This commit includes:
- EuroBERT model implementation for GGUF conversion
- C++ backend support for EuroBERT architecture
- Unintended autoformatting changes to Python files

Saving before reverting formatting-only changes.
@maximilianwerk
feat: add back eos assert when not last token pooling
9115f3a
@maximilianwerk
feat: removed duplicated code and cleanup
90fdbeb
@maximilianwerk

maximilianwerk commented Feb 24, 2026

Copy link
Copy Markdown
Contributor Author

Thanks a lot for the super quick reaction and all your comments. I adjusted the code accordingly.

CISC
CISC reviewed Feb 24, 2026
View reviewed changes
Comment thread convert_hf_to_gguf.py Outdated
Comment thread convert_hf_to_gguf.py Outdated
Comment thread convert_hf_to_gguf.py Outdated
CISC
CISC reviewed Feb 24, 2026
View reviewed changes
Comment thread convert_hf_to_gguf_update.py Outdated
CISC
CISC approved these changes Feb 25, 2026
View reviewed changes
Comment thread convert_hf_to_gguf.py Outdated
Comment thread src/models/eurobert.cpp Outdated
Comment thread src/models/eurobert.cpp Outdated
Comment thread src/models/eurobert.cpp Outdated
Comment thread src/models/eurobert.cpp Outdated
Comment thread src/models/eurobert.cpp Outdated
Comment thread src/models/eurobert.cpp Outdated
Comment thread src/models/eurobert.cpp Outdated
Comment thread src/models/eurobert.cpp Outdated
Comment thread tests/test-tokenizer-0.sh
@github-actions github-actions Bot added the testing Everything test related label Feb 25, 2026
@maximilianwerk

maximilianwerk commented Feb 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

When I ran the failing test locally, the tests that failed in CI passed, but another one was failing (test-tokenizers-ggml-vocabs - complaining about missing vocab files for quite some models, but not the newly added). Are these tests flaky and thus expected to fail here and there?

@CISC

CISC commented Feb 26, 2026

Copy link
Copy Markdown
Member

When I ran the failing test locally, the tests that failed in CI passed, but another one was failing (test-tokenizers-ggml-vocabs - complaining about missing vocab files for quite some models, but not the newly added). Are these tests flaky and thus expected to fail here and there?

If you're talking about ggml-ci-x64-cpu-low-perf, ignore it, it's just a random ccache problem, everything is ok, will merge now.

Regarding test-tokenizers-ggml-vocabs you might not have installed and initialized git-lfs? Then the files are not downloaded properly, either way, not important. :)

@CISC
CISC merged commit 66287bd into ggml-org:master Feb 26, 2026
81 of 82 checks passed
@maximilianwerk
maximilianwerk deleted the feat-jina-v5-text branch February 26, 2026 15:25
@maximilianwerk

maximilianwerk commented Feb 26, 2026

Copy link
Copy Markdown
Contributor Author

Amazing. Thanks again for the very pleasant experience. You did a great job in reviewing! ❤️

@rick-github rick-github mentioned this pull request Mar 5, 2026
Closed
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
@maximilianwerk
model : add Jina Embeddings v5 Nano (partial EuroBERT) support (ggml-…
1d4a3c8 
…org#19826)

* WIP: Add EuroBERT support with autoformatting changes

This commit includes:
- EuroBERT model implementation for GGUF conversion
- C++ backend support for EuroBERT architecture
- Unintended autoformatting changes to Python files

Saving before reverting formatting-only changes.

* feat: add back eos assert when not last token pooling

* feat: removed duplicated code and cleanup

* feat: removed not working architectures and unnecessary check

* fix: typo

* fix: dynamic pooling config

* feat: added an example model for eurobert

* feat: proper llama-vocab implementation for jina-v5

* fix: removed unnecessary comments
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
@maximilianwerk
model : add Jina Embeddings v5 Nano (partial EuroBERT) support (ggml-…
916d423 
…org#19826)

* WIP: Add EuroBERT support with autoformatting changes

This commit includes:
- EuroBERT model implementation for GGUF conversion
- C++ backend support for EuroBERT architecture
- Unintended autoformatting changes to Python files

Saving before reverting formatting-only changes.

* feat: add back eos assert when not last token pooling

* feat: removed duplicated code and cleanup

* feat: removed not working architectures and unnecessary check

* fix: typo

* fix: dynamic pooling config

* feat: added an example model for eurobert

* feat: proper llama-vocab implementation for jina-v5

* fix: removed unnecessary comments
fewtarius pushed a commit to fewtarius/CachyLLama that referenced this pull request May 30, 2026
@maximilianwerk
model : add Jina Embeddings v5 Nano (partial EuroBERT) support (ggml-…
816bdef 
…org#19826)

* WIP: Add EuroBERT support with autoformatting changes

This commit includes:
- EuroBERT model implementation for GGUF conversion
- C++ backend support for EuroBERT architecture
- Unintended autoformatting changes to Python files

Saving before reverting formatting-only changes.

* feat: add back eos assert when not last token pooling

* feat: removed duplicated code and cleanup

* feat: removed not working architectures and unnecessary check

* fix: typo

* fix: dynamic pooling config

* feat: added an example model for eurobert

* feat: proper llama-vocab implementation for jina-v5

* fix: removed unnecessary comments
fukuro-kun pushed a commit to fukuro-kun/fukuro-llama-cpp-turboquant that referenced this pull request Jul 5, 2026
@maximilianwerk
model : add Jina Embeddings v5 Nano (partial EuroBERT) support (ggml-…
ca4d235 
…org#19826)

* WIP: Add EuroBERT support with autoformatting changes

This commit includes:
- EuroBERT model implementation for GGUF conversion
- C++ backend support for EuroBERT architecture
- Unintended autoformatting changes to Python files

Saving before reverting formatting-only changes.

* feat: add back eos assert when not last token pooling

* feat: removed duplicated code and cleanup

* feat: removed not working architectures and unnecessary check

* fix: typo

* fix: dynamic pooling config

* feat: added an example model for eurobert

* feat: proper llama-vocab implementation for jina-v5

* fix: removed unnecessary comments
MrLordCat referenced this pull request in MrLordCat/llama.cpp-rdna-lab Jul 16, 2026
@maximilianwerk
model : add Jina Embeddings v5 Nano (partial EuroBERT) support (#19826)
9a1762e 
* WIP: Add EuroBERT support with autoformatting changes

This commit includes:
- EuroBERT model implementation for GGUF conversion
- C++ backend support for EuroBERT architecture
- Unintended autoformatting changes to Python files

Saving before reverting formatting-only changes.

* feat: add back eos assert when not last token pooling

* feat: removed duplicated code and cleanup

* feat: removed not working architectures and unnecessary check

* fix: typo

* fix: dynamic pooling config

* feat: added an example model for eurobert

* feat: proper llama-vocab implementation for jina-v5

* fix: removed unnecessary comments
zommiommy pushed a commit to zommiommy/llama.cpp that referenced this pull request Aug 18, 2026
@maximilianwerk @zommiommy
model : add Jina Embeddings v5 Nano (partial EuroBERT) support (ggml-…
93a8f08 
…org#19826)

* WIP: Add EuroBERT support with autoformatting changes

This commit includes:
- EuroBERT model implementation for GGUF conversion
- C++ backend support for EuroBERT architecture
- Unintended autoformatting changes to Python files

Saving before reverting formatting-only changes.

* feat: add back eos assert when not last token pooling

* feat: removed duplicated code and cleanup

* feat: removed not working architectures and unnecessary check

* fix: typo

* fix: dynamic pooling config

* feat: added an example model for eurobert

* feat: proper llama-vocab implementation for jina-v5

* fix: removed unnecessary comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CISC CISC CISC approved these changes
@ggerganov ggerganov Awaiting requested review from ggerganov ggerganov is a code owner

Assignees

No one assigned

Labels

examples model Model specific python python script changes testing Everything test related

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@maximilianwerk @CISC