Nature is infinitely resolution-free. In the context of this reality, existing diffusion models, such as Diffusion Transformers, often face challenges when processing image resolutions outside of their trained domain. To overcome this limitation, we present the Flexible Vision Transformer (FiT), a transformer architecture specifically designed for generating images with unrestricted resolutions and aspect ratios. Unlike traditional methods that perceive images as static-resolution grids, FiT conceptualizes images as sequences of dynamically-sized tokens. This perspective enables a flexible training strategy that effortlessly adapts to diverse aspect ratios during both training and inference phases, thus promoting resolution generalization and eliminating biases induced by image cropping. Enhanced by a meticulously adjusted network structure and the integration of training-free extrapolation techniques, FiT exhibits remarkable flexibility in resolution extrapolation generation. Comprehensive experiments demonstrate the exceptional performance of FiT across a broad range of resolutions, showcasing its effectiveness both within and beyond its training resolution distribution. Repository available at https://github.com/whlzy/FiT.

翻译：自然界的图像本质上是无限分辨率的。在此背景下，现有扩散模型（如扩散Transformer）在处理训练域之外的图像分辨率时往往面临挑战。为突破这一限制，我们提出柔性视觉Transformer（FiT）——一种专为生成任意分辨率和宽高比图像设计的Transformer架构。与传统方法将图像视为固定分辨率网格不同，FiT将图像概念化为动态尺寸令牌序列。这一视角使训练策略具有灵活性，能够在训练和推理阶段轻松适配多样的宽高比，从而促进分辨率泛化并消除由图像裁剪引起的偏差。通过精心调整的网络结构与无需训练的 extrapolation 技术相结合，FiT在分辨率外推生成方面展现出卓越的灵活性。全面的实验表明，FiT在广泛的分辨率范围内均表现出色，其有效性在其训练分辨率分布内外均得到验证。代码仓库详见 https://github.com/whlzy/FiT。