While recent advances in generative latent spaces have driven substantial progress in single-image generation, the optimal latent space for novel view synthesis (NVS) remains largely unexplored. In particular, NVS requires geometrically consistent generation across viewpoints, but existing approaches typically operate in a view-independent VAE latent space. In this paper, we propose Geometric Latent Diffusion (GLD), a framework that repurposes the geometrically consistent feature space of geometric foundation models as the latent space for multi-view diffusion. We show that these features not only support high-fidelity RGB reconstruction but also encode strong cross-view geometric correspondences, providing a well-suited latent space for NVS. Our experiments demonstrate that GLD outperforms both VAE and RAE on 2D image quality and 3D consistency metrics, while accelerating training by more than 4.4x compared to the VAE latent space. Notably, GLD remains competitive with state-of-the-art methods that leverage large-scale text-to-image pretraining, despite training its diffusion model from scratch without such generative pretraining.

翻译：尽管生成潜在空间的最新进展推动了单图像生成的显著进步，但用于新视角合成（NVS）的最优潜在空间仍 largely 未被探索。特别是，NVS 需要在不同视角间实现几何一致的生成，但现有方法通常在与视角无关的 VAE 潜在空间中操作。在本文中，我们提出几何潜在扩散（GLD）框架，该框架将几何基础模型的几何一致特征空间重新用作多视角扩散的潜在空间。我们表明，这些特征不仅支持高保真度 RGB 重建，而且编码了强大的跨视角几何对应关系，为 NVS 提供了高度适用的潜在空间。实验表明，GLD 在二维图像质量和三维一致性指标上均优于 VAE 和 RAE，同时相较于 VAE 潜在空间，训练速度提升超过 4.4 倍。值得注意的是，尽管 GLD 的扩散模型是从零开始训练且未使用此类生成预训练，但它仍能与依赖大规模文本到图像预训练的最新方法竞争。