From Gradients to Capabilities in Multi-Teacher On-Policy Distillation
An analysis of multi-teacher on-policy distillation compares gradients and optimizer updates across models to evaluate domain teacher signal integration.
An analysis of multi-teacher on-policy distillation compares gradients and optimizer updates across models to evaluate domain teacher signal integration.