Conventional depth-free multi-view datasets are captured using a moving monocular camera without metric calibration. The scales of camera positions in this monocular setting are ambiguous. Previous methods have acknowledged scale ambiguity in multi-view data via various ad-hoc normalization pre-processing steps, but have not directly analyzed the effect of incorrect scene scales on their application. In this paper, we seek to understand and address the effect of scale ambiguity when used to train generative novel view synthesis methods (GNVS). In GNVS, new views of a scene or object can be minimally synthesized given a single image and are, thus, unconstrained, necessitating the use of generative methods. The generative nature of these models captures all aspects of uncertainty, including any uncertainty of scene scales, which act as nuisance variables for the task. We study the effect of scene scale ambiguity in GNVS when sampled from a single image by isolating its effect on the resulting models and, based on these intuitions, define new metrics that measure the scale inconsistency of generated views. We then propose a framework to estimate scene scales jointly with the GNVS model in an end-to-end fashion. Empirically, we show that our method reduces the scale inconsistency of generated views without the complexity or downsides of previous scale normalization methods. Further, we show that removing this ambiguity improves generated image quality of the resulting GNVS model.

翻译：传统的无深度多视角数据集是通过移动的单目相机采集的，无需进行度量校准。在这种单目设置下，相机位置的尺度是模糊的。先前的方法通过各种临时性的归一化预处理步骤承认了多视角数据中的尺度模糊性，但并未直接分析错误场景尺度对其应用的影响。在本文中，我们旨在理解并解决尺度模糊性在用于训练生成式新视角合成方法时所产生的影响。在生成式新视角合成中，给定单张图像即可最小化地合成场景或物体的新视角，因此其不受约束，需要使用生成式方法。这些模型的生成性质捕捉了所有方面的不确定性，包括场景尺度的任何不确定性，这些不确定性对于任务而言是干扰变量。我们通过分离其影响来研究从单张图像采样时场景尺度模糊性在生成式新视角合成中的效应，并基于这些直观理解，定义了衡量生成视角尺度不一致性的新指标。随后，我们提出了一个框架，以端到端的方式与生成式新视角合成模型联合估计场景尺度。实验表明，我们的方法降低了生成视角的尺度不一致性，且避免了先前尺度归一化方法的复杂性或缺点。此外，我们还证明消除这种模糊性提高了所得生成式新视角合成模型的生成图像质量。