MISA: Memory-Efficient LLM Optimization

MISA is a memory-efficient optimization method for large language models. It uses module-wise importance sampling to reduce the memory demands of pre-training and fine-tuning while retaining the optimization signal from selected modules.

The work was accepted by NeurIPS 2025.

• NeurIPS paper
• arXiv
• Code
• Project page