We present a 3D modeling method which enables end-users to refine or detailize 3D shapes using machine learning, expanding the capabilities of AI-assisted 3D content creation. Given a coarse voxel shape (e.g., one produced with a simple box extrusion tool or via generative modeling), a user can directly "paint" desired target styles representing compelling geometric details, from input exemplar shapes, over different regions of the coarse shape. These regions are then up-sampled into high-resolution geometries which adhere with the painted styles. To achieve such controllable and localized 3D detailization, we build on top of a Pyramid GAN by making it masking-aware. We devise novel structural losses and priors to ensure that our method preserves both desired coarse structures and fine-grained features even if the painted styles are borrowed from diverse sources, e.g., different semantic parts and even different shape categories. Through extensive experiments, we show that our ability to localize details enables novel interactive creative workflows and applications. Our experiments further demonstrate that in comparison to prior techniques built on global detailization, our method generates structure-preserving, high-resolution stylized geometries with more coherent shape details and style transitions.

翻译：我们提出了一种三维建模方法，使得终端用户能够利用机器学习对三维形状进行精细化或细节化处理，从而扩展了AI辅助三维内容创作的能力。给定一个粗糙的体素形状（例如通过简单的立方体挤出工具或生成式建模产生），用户可以直接在粗糙形状的不同区域上，从输入示例形状中“绘制”出代表引人注目的几何细节的目标风格。这些区域随后被上采样为符合所绘制风格的高分辨率几何体。为实现这种可控且局部化的三维细节化，我们在金字塔GAN的基础上进行改进，使其具备掩码感知能力。我们设计了新颖的结构损失函数与先验约束，以确保即使绘制的风格来源于不同来源（例如不同的语义部件甚至不同的形状类别），我们的方法仍能同时保留所需的粗粒度结构与细粒度特征。通过大量实验，我们证明了局部化细节的能力能够支持新颖的交互式创意工作流程与应用。我们的实验进一步表明，与基于全局细节化的现有技术相比，本方法能够生成结构保持性更好、细节更连贯且风格过渡更自然的高分辨率风格化几何体。