Sparse attention is becoming increasingly important for serving large language models (LLMs) as generation lengths continue to grow. However, deploying and evaluating new sparse attention algorithms at scale remains highly engineering-intensive, slowing both human researchers and AI agents in exploring the sparse attention design. To address this challenge, we present Vortex, a system that combines a Python-embedded frontend language atop a page-centric tensor abstraction for expressing a broad range of sparse attention algorithms, with an efficient backend tightly integrated into modern LLM serving stacks. Vortex enables rapid prototyping, deployment, and evaluation of sparse attention algorithms, effectively translating their theoretical efficiency gains into real-world throughput improvements. As a result, Vortex substantially accelerates the design and iteration of sparse attention algorithms. First, AI agents use Vortex to automatically generate and refine diverse algorithms, the best reaching up to $3.46\times$ higher throughput than full attention while preserving accuracy. Second, Vortex extends sparse attention to emerging architectures and very large models that are otherwise hard to experiment with, reaching up to $4.7\times$ higher throughput on the MLA-based GLM-4.7-Flash and $1.37\times$ on the 229B-parameter MiniMax-M2.7 on NVIDIA B200 GPUs.

翻译：随着生成文本长度的持续增长，稀疏注意力在大型语言模型（LLM）服务中日趋重要。然而，大规模部署和评估新型稀疏注意力算法依然高度依赖工程实现，既阻碍了人类研究者也限制了AI Agent探索稀疏注意力设计空间。为解决这一挑战，我们提出Vortex——该系统在面向页面的张量抽象层之上，结合Python嵌入式前端语言，可表达广泛类别的稀疏注意力算法，同时配备与现代LLM服务栈紧密集成的高效后端引擎。Vortex支持稀疏注意力算法的快速原型构建、部署与评估，有效将其理论效率提升转化为实际吞吐量改进。实验表明，Vortex显著加速了稀疏注意力算法的设计迭代：首先，AI Agent利用Vortex自动生成并优化多种算法，最优方案在保持精度的前提下实现全注意力机制3.46倍以上的吞吐量提升；其次，Vortex将稀疏注意力扩展到新兴架构及难以进行实验的超大规模模型上——在基于MLA的GLM-4.7-Flash模型上实现4.7倍吞吐量提升，在配备NVIDIA B200 GPU的229B参数MiniMax-M2.7模型上实现1.37倍吞吐量提升。