We propose Stratified Image Transformer(StraIT), a pure non-autoregressive(NAR) generative model that demonstrates superiority in high-quality image synthesis over existing autoregressive(AR) and diffusion models(DMs). In contrast to the under-exploitation of visual characteristics in existing vision tokenizer, we leverage the hierarchical nature of images to encode visual tokens into stratified levels with emergent properties. Through the proposed image stratification that obtains an interlinked token pair, we alleviate the modeling difficulty and lift the generative power of NAR models. Our experiments demonstrate that StraIT significantly improves NAR generation and out-performs existing DMs and AR methods while being order-of-magnitude faster, achieving FID scores of 3.96 at 256*256 resolution on ImageNet without leveraging any guidance in sampling or auxiliary image classifiers. When equipped with classifier-free guidance, our method achieves an FID of 3.36 and IS of 259.3. In addition, we illustrate the decoupled modeling process of StraIT generation, showing its compelling properties on applications including domain transfer.

翻译：我们提出分层图像Transformer（StraIT），这是一种纯非自回归（NAR）生成模型，在高质量图像合成方面展现出优于现有自回归（AR）模型和扩散模型（DMs）的性能。针对现有视觉分词器对视觉特征利用不足的问题，我们利用图像的分层特性，将视觉标记编码为具有涌现特性的分层级别。通过所提出的图像分层方法获得相互关联的标记对，我们降低了建模难度并提升了NAR模型的生成能力。实验表明，StraIT显著改进了NAR生成性能，且相比现有DMs和AR方法实现数量级的速度提升——在ImageNet数据集上以256×256分辨率无需借助任何采样指导或辅助图像分类器即可达到3.96的FID分数。当采用无分类器指导时，我们的方法实现了3.36的FID和259.3的IS值。此外，我们展示了StraIT生成过程的解耦建模特性，揭示了其在域迁移等应用中令人瞩目的性能。