RoLA: Rotary-Positioned Low-Rank Linear Attention

Created on September 08, 2026

2026   ·   diffusion-transformers   video-generation   efficient-training   ·   research

RoLA introduces a rotary-positioned low-rank linear attention branch for efficient Diffusion Transformers. The design combines low-rank linear attention with rotary positional information for efficient visual generation.



Enjoy Reading This Article?

Here are some more articles you might like to read next:

  • SparkDiffusion: Mitigating the High-Sparsity Trap
  • CrossDistill: Balancing Quality and Diversity
  • RoPeSLR: Sparse-Low-Rank Attention for Diffusion Transformers
  • CR-Net: Cross-Layer Low-Rank Structure
  • Mixture of Distributions Matters