Transfer learning of large-scale Text-to-Image (T2I) models has recently shown impressive potential for Novel View Synthesis (NVS) of diverse objects from a single image. While previous methods typically train large models on multi-view datasets for NVS, fine-tuning the whole parameters of T2I models not only demands a high cost but also reduces the generalization capacity of T2I models in generating diverse images in a new domain. In this study, we propose an effective method, dubbed NVS-Adapter, which is a plug-and-play module for a T2I model, to synthesize novel multi-views of visual objects while fully exploiting the generalization capacity of T2I models. NVS-Adapter consists of two main components; view-consistency cross-attention learns the visual correspondences to align the local details of view features, and global semantic conditioning aligns the semantic structure of generated views with the reference view. Experimental results demonstrate that the NVS-Adapter can effectively synthesize geometrically consistent multi-views and also achieve high performance on benchmarks without full fine-tuning of T2I models. The code and data are publicly available in ~\href{https://postech-cvlab.github.io/nvsadapter/}{https://postech-cvlab.github.io/nvsadapter/}.

翻译：大规模文本到图像（T2I）模型的迁移学习近期在基于单张图像合成多样物体的新视角（NVS）方面展现出显著潜力。尽管现有方法通常在多视角数据集上训练大型模型以完成NVS任务，但微调T2I模型全部参数不仅成本高昂，还会降低模型在新领域生成多样化图像的泛化能力。本研究提出一种名为NVS-Adapter的高效方法——一种面向T2I模型的即插即用模块——在充分利用T2I模型泛化能力的同时，合成视觉物体的多视角新视图。NVS-Adapter包含两个核心组件：视图一致性交叉注意力机制学习视觉对应关系以对齐视角特征的局部细节，全局语义约束则使生成视图的语义结构与参考视图保持一致。实验结果表明，NVS-Adapter无需对T2I模型进行全参数微调，即可有效合成几何一致的多视角图像，并在基准测试中取得高性能表现。相关代码与数据已在\href{https://postech-cvlab.github.io/nvsadapter/}{https://postech-cvlab.github.io/nvsadapter/}公开。