BeNice! Closing Looping's Inference-Memory Gap via KV-Sharing
This paper introduces KV-sharing to mitigate the linear memory growth of looped architectures during inference.
This paper introduces KV-sharing to mitigate the linear memory growth of looped architectures during inference.