Skip to content

on_boarding.md

Latest commit

 

History

History
76 lines (45 loc) · 4.35 KB

on_boarding.md

File metadata and controls

76 lines (45 loc) · 4.35 KB
graph LR
    Embedding_Models["Embedding Models"]
    Reranker_Models["Reranker Models"]
    Model_Preprocessing["Model Preprocessing"]
    Evaluation_Framework["Evaluation Framework"]
    RAG_Integration["RAG Integration"]
    Model_Preprocessing -- "Preprocesses input data for" --> Embedding_Models
    Model_Preprocessing -- "Preprocesses input data for" --> Reranker_Models
    Embedding_Models -- "Provides embeddings to" --> Reranker_Models
    Reranker_Models -- "Submits reranking results for performance evaluation to" --> Evaluation_Framework
    Evaluation_Framework -- "Provides feedback on model performance to" --> Reranker_Models
    RAG_Integration -- "Utilizes" --> Reranker_Models
    RAG_Integration -- "Prepares and feeds RAG-specific data for evaluation to" --> Evaluation_Framework
    click Embedding_Models href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/BCEmbedding/Embedding_Models.md" "Details"
    click Reranker_Models href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/BCEmbedding/Reranker_Models.md" "Details"
    click RAG_Integration href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/BCEmbedding/RAG_Integration.md" "Details"
Loading

CodeBoardingDemoContact

Details

The BCEmbedding project is designed as a modular ML toolkit primarily focused on providing high-performance embedding and reranking capabilities for Retrieval Augmented Generation (RAG) and semantic search, especially in bilingual and crosslingual contexts.

Embedding Models [Expand]

Core component for generating dense vector representations of text, foundational for retrieval tasks.

Related Classes/Methods:

Reranker Models [Expand]

Implements various models (bi-encoder, cross-encoder) to re-order retrieved documents based on relevance to a query.

Related Classes/Methods:

Model Preprocessing

Handles the preparation of raw input data, including tokenization and input merging, for consumption by both embedding and reranker models.

Related Classes/Methods:

Evaluation Framework

Orchestrates the evaluation of models, particularly rerankers, by loading datasets, computing performance metrics, and integrating with specialized evaluators.

Related Classes/Methods:

RAG Integration [Expand]

Provides interfaces and utilities for seamlessly integrating BCEmbedding's models into RAG pipelines built with frameworks like Langchain and LlamaIndex, including data extraction and retrieval logic.

Related Classes/Methods:

FAQ