The Geometry of Language Model Logits: Structure and Departures
An empirical study identifies a consistent log-linear relation in ordered vocabulary-head logits across autoregressive language model architectures and sca
An empirical study identifies a consistent log-linear relation in ordered vocabulary-head logits across autoregressive language model architectures and sca