TACO: Ternary Column-wise Optimizer for LLM Fine-Tuning
TACO offers a memory-efficient optimizer for fine-tuning large language models by using ternary absolute-max column-wise one-sparse updates.
TACO offers a memory-efficient optimizer for fine-tuning large language models by using ternary absolute-max column-wise one-sparse updates.