Skip to content

Result_Management_Reporting.md

Latest commit

 

History

History
45 lines (29 loc) · 4.72 KB

Result_Management_Reporting.md

File metadata and controls

45 lines (29 loc) · 4.72 KB
graph LR
    src_lighteval_pipeline_Pipeline["src.lighteval.pipeline.Pipeline"]
    src_lighteval_logging_evaluation_tracker_EvaluationTracker["src.lighteval.logging.evaluation_tracker.EvaluationTracker"]
    src_lighteval_pipeline_Pipeline -- "delegates results to" --> src_lighteval_logging_evaluation_tracker_EvaluationTracker
Loading

CodeBoardingDemoContact

Details

This subsystem is responsible for the persistence, loading, and presentation of evaluation results, ensuring that detailed responses and aggregated scores are managed effectively and can be integrated with external platforms for visualization and sharing.

src.lighteval.pipeline.Pipeline

This component orchestrates the final stages of the evaluation pipeline, focusing on preparing, computing, and initiating the display and saving of results. It acts as the high-level controller for ensuring results are ready for output and presentation. Its responsibilities include post-processing outputs, computing final metrics, and triggering the persistence and reporting mechanisms.

Related Classes/Methods:

src.lighteval.logging.evaluation_tracker.EvaluationTracker

This component manages the actual storage, loading, and external integration of evaluation results and detailed responses. It handles the low-level persistence mechanisms (e.g., saving to disk) and interfaces with various external reporting platforms such as Hugging Face Hub, TensorBoard, and Weights & Biases. It ensures the long-term availability and shareability of evaluation outcomes.

Related Classes/Methods:

FAQ