This paper presents a method to reconstruct high-quality textured 3D models from single images. Current methods rely on datasets with expensive annotations; multi-view images and their camera parameters. Our method relies on GAN generated multi-view image datasets which have a negligible annotation cost. However, they are not strictly multi-view consistent and sometimes GANs output distorted images. This results in degraded reconstruction qualities. In this work, to overcome these limitations of generated datasets, we have two main contributions which lead us to achieve state-of-the-art results on challenging objects: 1) A robust multi-stage learning scheme that gradually relies more on the models own predictions when calculating losses, 2) A novel adversarial learning pipeline with online pseudo-ground truth generations to achieve fine details. Our work provides a bridge from 2D supervisions of GAN models to 3D reconstruction models and removes the expensive annotation efforts. We show significant improvements over previous methods whether they were trained on GAN generated multi-view images or on real images with expensive annotations. Please visit our web-page for 3D visuals: https://research.nvidia.com/labs/adlr/progressive-3d-learning

翻译：本文提出了一种从单张图像重建高质量纹理化3D模型的方法。现有方法依赖带有昂贵标注的数据集（多视角图像及其相机参数），而我们的方法利用GAN生成的多视角图像数据集，其标注成本可忽略不计。然而，这些生成数据并非严格意义上的多视角一致，且GAN有时会输出畸变图像，导致重建质量下降。为克服生成数据集的这些局限性，本文做出两项主要贡献，从而在挑战性物体上取得了最先进的结果：1）一种鲁棒的多阶段学习方案，在计算损失时逐步更多地依赖模型自身的预测；2）一种结合在线伪真实值生成的新型对抗学习流程，以实现精细细节的重建。我们的工作为从GAN模型的2D监督到3D重建模型搭建了桥梁，消除了昂贵的标注负担。实验表明，无论相较于基于GAN生成多视角图像训练的方法，还是基于带有昂贵标注的真实图像训练的方法，我们的方法均有显著提升。3D效果展示请访问我们的网页：https://research.nvidia.com/labs/adlr/progressive-3d-learning