Sonar-MoE: In-Backward Expert Updates for MoE Training
Sonar-MoE updates expert tensors during the backward pass to reduce GPU memory usage in Mixture-of-Experts training.
Sonar-MoE updates expert tensors during the backward pass to reduce GPU memory usage in Mixture-of-Experts training.