Exact Retrieval Across 2 Million Tokens with Ring-Sparse Attention

Abstract & Key Findings

Many long-context models advertise million-token windows on paper but suffer severe reasoning degradation ("context rot") once inputs exceed 64,000 tokens. We introduce Ring-Sparse Attention (RSA), the core long-context mechanism powering Alchemax-5.

By dynamically routing global attention heads to high-salience semantic anchors while maintaining local sliding windows in SRAM, RSA achieves 99.8% multi-needle retrieval and complex cross-document synthesis across the full 2,000,000-token window at 3.4x lower memory bandwidth.