MISA
Memory-efficient LLM optimization with module-wise importance sampling.
MISA reduces the memory requirements of large language model pre-training and fine-tuning through module-wise importance sampling.
Memory-efficient LLM optimization with module-wise importance sampling.
MISA reduces the memory requirements of large language model pre-training and fine-tuning through module-wise importance sampling.