MuonGS: Pretraining LLMs with Relaxed Weight-Norm Control
This study analyzes norm control in Muon over the entire training trajectory using the accumulated effective learning rate to improve language model pretra
This study analyzes norm control in Muon over the entire training trajectory using the accumulated effective learning rate to improve language model pretra