Accurate and efficient discrete video tokenization is essential for long video sequences processing. Yet, the inherent complexity and variable information density of videos present a significant bottleneck for current tokenizers, which rigidly compress all content at a fixed rate, leading to redundancy or information loss. Drawing inspiration from Shannon's information theory, this paper introduces InfoTok, a principled framework for adaptive video tokenization. We rigorously prove that existing data-agnostic training methods are suboptimal in representation length, and present a novel evidence lower bound (ELBO)-based algorithm that approaches theoretical optimality. Leveraging this framework, we develop a transformer-based adaptive compressor that enables adaptive tokenization. Empirical results demonstrate state-of-the-art compression performance, saving 20% tokens without influence on performance, and achieving 2.3x compression rates while still outperforming prior heuristic adaptive approaches. By allocating tokens according to informational richness, InfoTok enables a more compressed yet accurate tokenization for video representation, offering valuable insights for future research.

翻译：精确且高效的离散视频分词对于长视频序列处理至关重要。然而，视频固有的复杂性和可变的信息密度对当前分词器构成了显著瓶颈，这些分词器以固定速率对所有内容进行刚性压缩，导致冗余或信息丢失。受香农信息论的启发，本文提出了InfoTok，一个用于自适应视频分词的原则性框架。我们严格证明了现有数据无关训练方法在表示长度上是次优的，并提出了一种新颖的基于证据下界（ELBO）的算法，该算法逼近理论最优性。利用此框架，我们开发了一种基于Transformer的自适应压缩器，实现了自适应分词。实证结果表明了最先进的压缩性能，在不影响性能的情况下节省了20%的token，并实现了2.3倍的压缩率，同时仍优于先前的启发式自适应方法。通过根据信息丰富度分配token，InfoTok为视频表示提供了一种更压缩且更精确的分词方案，为未来研究提供了有价值的见解。