Depth Memory in Neural Networks
This research explores expressive depth aggregation mechanisms like mHC and AttnRes to improve language modeling performance at scale.
This research explores expressive depth aggregation mechanisms like mHC and AttnRes to improve language modeling performance at scale.