graph LR
Evaluation_Benchmarking_CLI_Orchestrator["Evaluation & Benchmarking CLI Orchestrator"]
Deep_Learning_Model_Evaluation_Core["Deep Learning Model Evaluation Core"]
Traditional_Codec_Benchmarking_Core["Traditional Codec Benchmarking Core"]
Metric_Calculation_Engine["Metric Calculation Engine"]
Evaluation_Benchmarking_CLI_Orchestrator -- "invokes" --> Deep_Learning_Model_Evaluation_Core
Evaluation_Benchmarking_CLI_Orchestrator -- "invokes" --> Traditional_Codec_Benchmarking_Core
Deep_Learning_Model_Evaluation_Core -- "calls" --> Metric_Calculation_Engine
Traditional_Codec_Benchmarking_Core -- "calls" --> Metric_Calculation_Engine
The Evaluation & Benchmarking Suite subsystem is designed to provide comprehensive tools for assessing the performance of both deep learning-based compression models and traditional codecs. It follows an "ML Toolkit/Library with an Evaluation Platform" pattern by separating orchestration, core evaluation logic, and metric computation.
This component serves as the primary command-line interface for the entire evaluation and benchmarking suite. It is responsible for parsing user arguments, configuring the evaluation environment, and dispatching control to the appropriate deep learning model evaluation or traditional codec benchmarking pipelines for both image and video data.
Related Classes/Methods:
compressai.utils.eval_model.__main__.main:386-510compressai.utils.bench.__main__.main:151-176compressai.utils.video.bench.__main__.main:320-360compressai.utils.video.eval_model.__main__.main:516-595
This component encapsulates the core logic for evaluating deep learning-based compression models. It handles tasks such as loading trained models, performing forward passes (inference) on input data (images or video frames), and preparing the model outputs for subsequent metric calculation. It abstracts away the specifics of image vs. video model inference.
Related Classes/Methods:
compressai.utils.eval_model.__main__.eval_model:210-264compressai.utils.video.eval_model.__main__.eval_model:233-300
This component is responsible for executing and evaluating traditional (non-deep learning) image and video compression codecs. It manages the interaction with external codec binaries, handles file input/output for compressed bitstreams, and prepares the decoded outputs for metric computation.
Related Classes/Methods:
This component provides a standardized set of functions for computing various quality metrics (e.g., PSNR, MS-SSIM) between original and reconstructed data. It supports both image and video data, including necessary pre-processing steps like color space transformations (e.g., YUV to RGB) for video frames.
Related Classes/Methods: