Core Foundation Layer
Base classes and foundational utilities including PreTrainedModel, PreTrainedConfig, output dataclasses, KV cache implementations, and core utilities that all other layers depend on.
12
Model Infrastructure Layer
Model building blocks including attention utilities, RoPE, initialization, masking, modular layers, and activation functions that define model architecture.
11
Tokenization Layer
Text tokenization utilities including base tokenizer classes, fast/slow tokenizers, SentencePiece integration, and tokenizer conversion tools.
7
Generation Layer
Text generation infrastructure including the main generate() method, generation configurations, and logits processors for controlling output quality.
3
Training Layer
Model training infrastructure including the Trainer class, training arguments, callbacks, optimization schedulers, and hyperparameter search.
11
Data Processing Layer
Data handling including data collators for batching and padding, data processors for specific datasets, and evaluation metrics.
3
Media Processing Layer
Image, audio, and video processing utilities including feature extraction, image transforms, and media-specific processing pipelines.
10
Pipelines and Integrations Layer
High-level task pipelines and external integrations including ASR pipelines, experiment tracking (W&B, TensorBoard), distributed training, and model export.
12
Utilities Layer
Supporting utilities including model cards, debugging tools, testing utilities, and general-purpose helpers that don't fit in other layers.
10