Scaling Theory of Multi-Epoch Pretraining and Ensembling
Researchers study how training epochs and ensemble size shape generalization when language model pretraining is data-bound.
Researchers study how training epochs and ensemble size shape generalization when language model pretraining is data-bound.