Existing 1D visual tokenizers for autoregressive (AR) generation largely follow the design principles of language modeling, as they are built directly upon transformers whose priors originate in language, yielding single-hierarchy latent tokens and treating visual data as flat sequential token streams. However, this language-like formulation overlooks key properties of vision, particularly the hierarchical and residual network designs that have long been essential for convergence and efficiency in visual models. To bring "vision" back to vision, we propose the Residual Tokenizer (ResTok), a 1D visual tokenizer that builds hierarchical residuals for both image tokens and latent tokens. The hierarchical representations obtained through progressively merging enable cross-level feature fusion at each layer, substantially enhancing representational capacity. Meanwhile, the semantic residuals between hierarchies prevent information overlap, yielding more concentrated latent distributions that are easier for AR modeling. Cross-level bindings consequently emerge without any explicit constraints. To accelerate the generation process, we further introduce a hierarchical AR generator that substantially reduces sampling steps by predicting an entire level of latent tokens at once rather than generating them strictly token-by-token. Extensive experiments demonstrate that restoring hierarchical residual priors in visual tokenization significantly improves AR image generation, achieving a gFID of 2.34 on ImageNet-256 with only 9 sampling steps. Code is available at https://github.com/Kwai-Kolors/ResTok.

翻译：现有的用于自回归（AR）生成的一维视觉分词器大多遵循语言建模的设计原则，因其直接构建于Transformer之上，而Transformer的先验源自语言领域，这导致其产生单一层次的潜在标记，并将视觉数据视为扁平的序列化标记流。然而，这种类语言的构建方式忽视了视觉的关键特性，尤其是长期以来对视觉模型的收敛与效率至关重要的层次化与残差网络设计。为使“视觉”回归视觉本质，我们提出残差分词器（ResTok），这是一种构建图像标记与潜在标记层次化残差的一维视觉分词器。通过逐层合并获得的层次化表示，使得每一层都能实现跨层级特征融合，从而显著提升表征能力。同时，层次间的语义残差避免了信息重叠，产生更集中的潜在分布，更易于进行AR建模。由此，跨层级绑定自然涌现，无需任何显式约束。为加速生成过程，我们进一步引入层次化AR生成器，通过一次性预测整个层级的潜在标记而非严格按标记逐个生成，大幅减少了采样步数。大量实验表明，在视觉分词中恢复层次化残差先验能显著提升AR图像生成效果，在ImageNet-256数据集上仅用9个采样步数即达到2.34的gFID分数。代码发布于 https://github.com/Kwai-Kolors/ResTok。