The rapid development of AR/VR brings tremendous demands for 3D content. While the widely-used Computer-Aided Design (CAD) method requires a time-consuming and labor-intensive modeling process, sketch-based 3D modeling offers a potential solution as a natural form of computer-human interaction. However, the sparsity and ambiguity of sketches make it challenging to generate high-fidelity content reflecting creators' ideas. Precise drawing from multiple views or strategic step-by-step drawings is often required to tackle the challenge but is not friendly to novice users. In this work, we introduce a novel end-to-end approach, Deep3DSketch+, which performs 3D modeling using only a single free-hand sketch without inputting multiple sketches or view information. Specifically, we introduce a lightweight generation network for efficient inference in real-time and a structural-aware adversarial training approach with a Stroke Enhancement Module (SEM) to capture the structural information to facilitate learning of the realistic and fine-detailed shape structures for high-fidelity performance. Extensive experiments demonstrated the effectiveness of our approach with the state-of-the-art (SOTA) performance on both synthetic and real datasets.

翻译：随着增强现实/虚拟现实技术的快速发展，三维内容需求呈现爆发式增长。尽管广泛使用的计算机辅助设计（CAD）方法能实现三维建模，但其建模过程耗时费力，而基于草图的三维建模因其人机交互的自然特性展现出潜在应用价值。然而，草图数据的稀疏性与歧义性使得生成反映创作者意图的高保真内容颇具挑战。现有方法通常需要通过多视角精确绘制或分步策略性绘制来应对该难题，但这对于新手用户并不友好。本文提出了一种新型端到端方法Deep3DSketch+，该方法仅需单张手绘草图即可完成三维建模，无需输入多张草图或视角信息。具体而言，我们设计了轻量级生成网络以实现实时高效推理，并引入结构感知对抗训练方法配合笔画增强模块（SEM），通过捕捉结构信息促进对真实精细形状结构的学习，从而实现高保真建模性能。大量实验表明，本方法在合成数据集与真实数据集上均达到最优（SOTA）性能。