World Wires · story 2592 · corroborated · 1 source(s)

Looped MoE: Flattening Experts and Untying Attention

A study explores combining looped transformers and mixture-of-experts architectures to improve expert utilization and parameter efficiency.

Open in the desk

Coverage

What this site indexes