Real-world applications often require a large gallery of 3D assets that share a consistent theme. While remarkable advances have been made in general 3D content creation from text or image, synthesizing customized 3D assets following the shared theme of input 3D exemplars remains an open and challenging problem. In this work, we present ThemeStation, a novel approach for theme-aware 3D-to-3D generation. ThemeStation synthesizes customized 3D assets based on given few exemplars with two goals: 1) unity for generating 3D assets that thematically align with the given exemplars and 2) diversity for generating 3D assets with a high degree of variations. To this end, we design a two-stage framework that draws a concept image first, followed by a reference-informed 3D modeling stage. We propose a novel dual score distillation (DSD) loss to jointly leverage priors from both the input exemplars and the synthesized concept image. Extensive experiments and user studies confirm that ThemeStation surpasses prior works in producing diverse theme-aware 3D models with impressive quality. ThemeStation also enables various applications such as controllable 3D-to-3D generation.
翻译:现实应用通常需要大量共享一致主题的三维资产库。尽管从文本或图像进行通用三维内容生成取得了显著进展,但根据输入三维示例的共享主题合成定制化三维资产仍然是一个具有挑战性的开放问题。本文提出主题站(ThemeStation),一种用于主题感知三维到三维生成的新型方法。主题站基于给定的少量示例合成定制化三维资产,追求两大目标:1)统一性——生成与示例在主题上保持一致的三维资产;2)多样性——生成具有高度变化的三维资产。为此,我们设计了一个两阶段框架,首先生成概念图像,随后进入参考信息引导的三维建模阶段。我们提出了一种新颖的双重分数蒸馏(DSD)损失函数,以联合利用输入示例与合成概念图像的先验知识。大量实验与用户研究证实,主题站在生成多样化的主题感知三维模型方面超越了先前工作,并展现出令人印象深刻的生成质量。主题站同时支持可控三维到三维生成等多种应用场景。