Diffusion and flow matching models have unlocked unprecedented capabilities for creative content creation, such as interactive image and streaming video generation. The growing demand for higher resolutions, frame rates, and context lengths, however, makes efficient generation increasingly challenging, as computational complexity grows quadratically with the number of generated tokens. Our work seeks to optimize the efficiency of the generation process in settings where the user's gaze location is known or can be estimated, for example, by using eye tracking. In these settings, we leverage the eccentricity-dependent acuity of human vision: while a user perceives very high-resolution visual information in a small region around their gaze location (the foveal region), the ability to resolve detail quickly degrades in the periphery of the visual field. Our approach starts with a mask modeling the foveated resolution to allocate tokens non-uniformly, assigning higher token density to foveal regions and lower density to peripheral regions. An image or video is generated in a mixed-resolution token setting, yielding results perceptually indistinguishable from full-resolution generation, while drastically reducing the token count and generation time. To this end, we develop a principled mechanism for constructing mixed-resolution tokens directly from high-resolution data, allowing a foveated diffusion model to be post-trained from an existing base model while maintaining content consistency across resolutions. We validate our approach through extensive analysis and a carefully designed user study, demonstrating the efficacy of foveation as a practical and scalable axis for efficient generation.

翻译：扩散模型与流匹配模型为创意内容生成（如交互式图像与流式视频生成）带来了前所未有的能力。然而，对更高分辨率、帧率和上下文长度的日益增长的需求，使得高效生成面临越来越大的挑战，因为计算复杂度随生成令牌数量的增加呈二次方增长。本研究旨在优化用户注视位置已知或可估计（例如通过眼动追踪）场景下的生成效率。在此类场景中，我们利用人类视觉的离心度依赖性敏锐度：虽然用户在注视点周围的小范围区域（中心凹区域）能感知到极高分辨率的视觉信息，但在视野外围，分辨细节的能力会迅速下降。我们的方法首先通过构建模拟中心凹分辨率的掩膜实现令牌的非均匀分配，为中心凹区域分配更高的令牌密度，为外围区域分配较低的密度。图像或视频在混合分辨率令牌设置下生成，其感知效果与全分辨率生成无显著差异，同时大幅减少令牌数量和生成时间。为此，我们开发了一种基于原则的机制，直接从高分辨率数据构建混合分辨率令牌，使得中心凹扩散模型能够在保持跨分辨率内容一致性的前提下，基于现有基础模型进行后训练。通过广泛分析与精心设计的用户研究，我们验证了该方法的有效性，证明了中心凹化作为实用且可扩展的高效生成维度的应用潜力。