graph LR
ModelLoader["ModelLoader"]
ModelInitializer["ModelInitializer"]
TextTokenizer["TextTokenizer"]
FeatureConverter["FeatureConverter"]
PytorchBERTFoundation["PytorchBERTFoundation"]
ModelLoader -- "provides configurations and weights to" --> ModelInitializer
PytorchBERTFoundation -- "offers foundational utilities to" --> ModelLoader
TextTokenizer -- "feeds tokenized output to" --> FeatureConverter
PytorchBERTFoundation -- "provides utilities to" --> TextTokenizer
The Model & Data Handler subsystem is responsible for the entire lifecycle of preparing models for inference or training, encompassing the loading of pre-trained model weights, the architectural initialization of these models, and the comprehensive transformation of raw text data into a format directly consumable by the models. It acts as the crucial interface between raw inputs (models and text) and the core distillation or inference processes.
Manages the loading of pre-trained model weights and configurations for both Teacher and Student models. This includes handling various sources (local files, cached paths, TensorFlow checkpoints) and converting weights to the PyTorch format, ensuring models are ready to be initialized with specific parameters.
Related Classes/Methods:
Constructs the architectural backbone of the Teacher and Student models. It often involves calling an initializer_builder to define how model weights are initialized and setting up internal model components, preparing the model for receiving loaded weights and subsequent operations.
Related Classes/Methods:
Converts raw text input into a sequence of tokens suitable for neural network models. This involves preprocessing steps like cleaning, accent stripping, and splitting on punctuation, adhering to the specific tokenization rules of the pre-trained models (e.g., BERT's WordPiece tokenization).
Related Classes/Methods:
Transforms tokenized text and other raw example data into numerical InputFeatures (e.g., input IDs, attention masks, segment IDs) that can be directly fed into the models. This component often includes task-specific logic for feature generation, adapting the tokenized data to the model's expected input format.
Related Classes/Methods:
examples.cmrc2018_example.processing.convert_examples_to_features:232-429examples.conll2003_example.utils_ner.convert_examples_to_features:90-210examples.mnli_example.utils_glue.convert_examples_to_features:378-488
Provides foundational utilities for BERT model loading and text tokenization. This component acts as a common base for BERT-specific operations, ensuring consistency and reusability across different parts of the model and data preparation pipeline.
Related Classes/Methods: