Is Weight Tying Beneficial for Decoder-Only LLMs Under DP-SGD?
This paper evaluates the impact of weight tying in decoder-only large language models when fine-tuned using Differentially Private Stochastic Gradient Desc
This paper evaluates the impact of weight tying in decoder-only large language models when fine-tuned using Differentially Private Stochastic Gradient Desc