CR-Net: Cross-Layer Low-Rank Structure

Created on January 26, 2026

2026   ·   llm   efficient-training   low-rank   ·   research

CR-Net explores cross-layer low-rank structure to scale parameter-efficient large language model training. The method shares and organizes low-rank information across layers rather than treating every layer independently.

The work was accepted by ICLR 2026.



Enjoy Reading This Article?

Here are some more articles you might like to read next:

  • SparkDiffusion: Mitigating the High-Sparsity Trap
  • CrossDistill: Balancing Quality and Diversity
  • RoLA: Rotary-Positioned Low-Rank Linear Attention
  • RoPeSLR: Sparse-Low-Rank Attention for Diffusion Transformers
  • Mixture of Distributions Matters