[Bug]: Unable to load 'Qdrant/bm25' on macOS python3.14

[Sirorezka]

What happened?

Code Crashes with 'Segmentation fault: 11'. Any suggestions on how to debug this?

from fastembed import SparseTextEmbedding

documents = [
    "You should stay, study and sprint.",
    "History can only prepare us to be surprised yet again.",
]

model = SparseTextEmbedding(model_name="Qdrant/bm25", cuda=False)

What Python version are you on? e.g. python --version

Python 3.14.3
fastembed v0.8.0
onnx v1.21.0
onnxruntime v1.24.4 (*)
optimum[onnx] v2.1.0

FastEmbed version

Python 3.14.3
fastembed v0.8.0
onnx v1.21.0
onnxruntime v1.24.4 (*)
optimum[onnx] v2.1.0

What os are you seeing the problem on?

Apple M4 Pro
macOS Tahoe 26.3.1 (25D2128)

Relevant stack traces and/or logs

Segmentation fault: 11

[shineSnow]

问题原因：QdrantVectorStore 初始化 enable_hybrid=True 时，内部用 local_files_only=True 调用 fastembed 加载 Qdrant/bm25 模型，导致它只查缓存不下载，缓存不存在就报错。

解决步骤：

1. 手动下载模型到缓存

在项目虚拟环境中执行：

随便找个地方跑一次就行

from fastembed.sparse import SparseTextEmbedding
model = SparseTextEmbedding(model_name='Qdrant/bm25')

模型文件会下载到 fastembed_cache/bm25 目录（路径见报错信息中的 /var/folders/5f/.../fastembed_cache/bm25）。

2. 保持 core/rag/vector_store.py 代码不变

即保留 enable_hybrid=True 和 fastembed_sparse_model="Qdrant/bm25"：

self.vector_store = QdrantVectorStore(
client=self.client,
collection_name=self.collection_name,
enable_hybrid=True,
fastembed_sparse_model="Qdrant/bm25",
batch_size=20
)

这样就解决了——模型在缓存里存在后，QdrantVectorStore 初始化时 local_files_only=True 就能找到，不会再抛异常。

[Sirorezka]

Hello, I tried following your guide, but still got same error:

# run_embed_data.py 

from langchain_qdrant import QdrantVectorStore
from fastembed.sparse import SparseTextEmbedding;
model = SparseTextEmbedding(model_name='Qdrant/bm25', cache_dir = "./bm25")
...

Here is the error:

(soglosov) (base) cab-wsm-0587372:approvals 22530325$ python run_embed_data.py 
german.txt: 1.36kB [00:00, 807kB/s]                                                            | 0/18 [00:00<?, ?it/s]
danish.txt: 100%|█████████████████████████████████████████████████████████████████████| 424/424 [00:00<00:00, 623kB/s]
config.json: 100%|█████████████████████████████████████████████████████████████████| 2.00/2.00 [00:00<00:00, 2.05kB/s]
dutch.txt: 100%|██████████████████████████████████████████████████████████████████████| 453/453 [00:00<00:00, 440kB/s]
arabic.txt: 6.35kB [00:00, 4.71MB/s]                                                        | 0.00/453 [00:00<?, ?B/s]
english.txt: 100%|███████████████████████████████████████████████████████████████████| 936/936 [00:00<00:00, 2.10MB/s]
french.txt: 100%|████████████████████████████████████████████████████████████████████| 813/813 [00:00<00:00, 1.75MB/s]
finnish.txt: 1.58kB [00:00, 3.22MB/s]                                                       | 0.00/813 [00:00<?, ?B/s]
greek.txt: 2.17kB [00:00, 28.5MB/s]                                                    | 1/18 [00:00<00:04,  3.57it/s]
hungarian.txt: 1.23kB [00:00, 12.3MB/s]
italian.txt: 1.65kB [00:00, 7.69MB/s]██████████████████▌                               | 9/18 [00:00<00:00, 21.20it/s]
romanian.txt: 1.91kB [00:00, 12.2MB/s]
portuguese.txt: 1.29kB [00:00, 22.6MB/s]
spanish.txt: 2.18kB [00:00, 27.7MB/s]
norwegian.txt: 100%|█████████████████████████████████████████████████████████████████| 851/851 [00:00<00:00, 16.4MB/s]
russian.txt: 1.24kB [00:00, 18.0MB/s]                                                       | 0.00/851 [00:00<?, ?B/s]
turkish.txt: 100%|███████████████████████████████████████████████████████████████████| 260/260 [00:00<00:00, 2.74MB/s]
swedish.txt: 100%|███████████████████████████████████████████████████████████████████| 559/559 [00:00<00:00, 11.4MB/s]
Fetching 18 files: 100%|██████████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 25.59it/s]
Segmentation fault: 11
(soglosov) (base) cab-wsm-0587372:approvals 22530325$ /Users/22530325/.local/share/uv/python/cpython-3.14.3-macos-aarch64-none/lib/python3.14/multiprocessing/resource_tracker.py:396: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown: {'/mp-fvhual9y'}
  warnings.warn(

[Sirorezka]

Basically the error is in py-rust-steamer

from py_rust_stemmers import SnowballStemmer
print("running steamer...")
SnowballStemmer("english")
print("done")

I get the following output:

running steamer...
Segmentation fault: 11

[Sirorezka]

Apple M4 Pro
macOs "26.4.1 (25E253)"
Python 3.14.3
conda 26.3.2
rustc 1.93.1 (01f6ddf75 2026-02-11)
maturin 1.11.5
cargo 1.93.1

Basically I ended up trying to build 'py_rust_stemmers' locally. But I wasn't been able to run 'SnowballStemmer' class despite any tries.

Solution that worked for me was to ask chatbot to update py_rust_stemmers dependency "pyo3" to "v0.28.3" and rewrite the rust code. After that with new library "pyo3" the problem was solved.

Here is new implementation of 'lib.rs' (from py_rust_stemmers)

use pyo3::prelude::*;
use rayon::prelude::*;
// Import the stemmer implementation from the rust-stemmers library
extern crate rust_stemmers;
use rust_stemmers::{Algorithm, Stemmer};

// Create a Python class to expose the stemmer functionality
#[pyclass]
pub struct SnowballStemmer {
    stemmer: Stemmer,
}

#[pymethods]
impl SnowballStemmer {
    #[new]
    fn new(lang: String) -> PyResult<Self> {  // Changed: &str -> String, added PyResult
        let algorithm = match lang.to_lowercase().as_str() {
            "arabic" => Algorithm::Arabic,
            "danish" => Algorithm::Danish,
            "dutch" => Algorithm::Dutch,
            "english" => Algorithm::English,
            "finnish" => Algorithm::Finnish,
            "french" => Algorithm::French,
            "german" => Algorithm::German,
            "greek" => Algorithm::Greek,
            "hungarian" => Algorithm::Hungarian,
            "italian" => Algorithm::Italian,
            "norwegian" => Algorithm::Norwegian,
            "portuguese" => Algorithm::Portuguese,
            "romanian" => Algorithm::Romanian,
            "russian" => Algorithm::Russian,
            "spanish" => Algorithm::Spanish,
            "swedish" => Algorithm::Swedish,
            "tamil" => Algorithm::Tamil,
            "turkish" => Algorithm::Turkish,
            _ => return Err(pyo3::exceptions::PyValueError::new_err(
                format!("Unsupported language: {}", lang)
            )),  // Changed: panic -> PyErr
        };
        let stemmer = Stemmer::create(algorithm);
        Ok(SnowballStemmer { stemmer })  // Wrapped in Ok
    }

    #[inline(always)]
    fn stem_word(&self, input: String) -> String {  // Changed: &str -> String
        self.stemmer.stem(&input).into_owned()  // Added & to convert String to &str
    }

    #[inline(always)]
    pub fn stem_words_parallel(&self, inputs: Vec<String>) -> Vec<String> {  // Changed: Vec<&str> -> Vec<String>
        inputs
            .into_par_iter()
            .map(|word| self.stemmer.stem(&word).into_owned())  // Added & to convert String to &str
            .collect()
    }

    #[inline(always)]
    pub fn stem_words(&self, inputs: Vec<String>) -> Vec<String> {
        inputs
            .into_iter()
            .map(|word| self.stemmer.stem(&word).into_owned())
            .collect()
    }
}

/// This module is required for the Python interpreter to access the Rust functions.
#[pymodule]
fn py_rust_stemmers(_py: Python, m: &Bound<'_, PyModule>) -> PyResult<()> {  // Changed: &PyModule -> &Bound<'_, PyModule>
    m.add_class::<SnowballStemmer>()?;
    Ok(())
}

Build script

conda install rust maturin 
pip uninstall py_rust_stemmers -y
git clone https://github.com/qdrant/py-rust-stemmers/
cd ./py-rust-stemmers
rm -rf ./target/ 
## edit '.src/lib.rs' file
cargo clean && maturin build --release
pip install --force-reinstall  ./target/wheels/*.whl
python -c "from py_rust_stemmers import SnowballStemmer; SnowballStemmer('english');"

python code:

from py_rust_stemmers import SnowballStemmer; 
stem = SnowballStemmer('english'); 
print(stem.stem_word('mushrooms')); 
print(stem.stem_words(['mushrooms','dogs']))

[DocG1]

I have the same error on macbook Air with M5

[anirbanbasu]

Note that this works with Python 3.13.3 but not with 3.14.x.

[joein]

Hello @Sirorezka @shineSnow @DocG1 @anirbanbasu

Sorry for the delay, py-rust-stemmers did not support python3.14, but now it is supported (as of py-rust-stemmers==0.1.8)
If you install fastembed into a fresh environment, it should fetch the latest version of py-rust-stemmers and just work.

[Sirorezka]

Thanks!!

[anirbanbasu]

@joein thanks, it works for me now.