Autoregressive (AR) video generative models rely on video tokenizers that compress pixels into discrete token sequences. The length of these token sequences is crucial for balancing reconstruction quality against downstream generation computational cost. Traditional video tokenizers apply a uniform token assignment across temporal blocks of different videos, often wasting tokens on simple, static, or repetitive segments while underserving dynamic or complex ones. To address this inefficiency, we introduce $\textbf{EVATok}$, a framework to produce $\textbf{E}$fficient $\textbf{V}$ideo $\textbf{A}$daptive $\textbf{Tok}$enizers. Our framework estimates optimal token assignments for each video to achieve the best quality-cost trade-off, develops lightweight routers for fast prediction of these optimal assignments, and trains adaptive tokenizers that encode videos based on the assignments predicted by routers. We demonstrate that EVATok delivers substantial improvements in efficiency and overall quality for video reconstruction and downstream AR generation. Enhanced by our advanced training recipe that integrates video semantic encoders, EVATok achieves superior reconstruction and state-of-the-art class-to-video generation on UCF-101, with at least 24.4% savings in average token usage compared to the prior state-of-the-art LARP and our fixed-length baseline.

翻译：自回归（AR）视频生成模型依赖于视频分词器，后者将像素压缩为离散的标记序列。这些标记序列的长度对于平衡重建质量与下游生成的计算成本至关重要。传统的视频分词器在不同视频的时间块上采用统一的标记分配方案，常常在简单、静态或重复的片段上浪费标记，而对动态或复杂的片段分配不足。为解决这一低效问题，我们提出了 $\textbf{EVATok}$ 框架，用于生成 $\textbf{E}$fficient $\textbf{V}$ideo $\textbf{A}$daptive $\textbf{Tok}$enizers（高效视频自适应分词器）。我们的框架为每个视频估计最优的标记分配，以实现最佳的质量-成本权衡；开发轻量级路由器以快速预测这些最优分配；并训练自适应分词器，使其根据路由器预测的分配方案对视频进行编码。我们证明，EVATok 在视频重建和下游 AR 生成的效率与整体质量方面均带来显著提升。通过我们集成了视频语义编码器的先进训练方案增强，EVATok 在 UCF-101 数据集上实现了卓越的重建效果和最先进的类别到视频生成性能，与先前最优的 LARP 以及我们固定长度的基线相比，平均标记使用量至少节省了 24.4%。