3D generation has witnessed significant advancements, yet efficiently producing high-quality 3D assets from a single image remains challenging. In this paper, we present a triplane autoencoder, which encodes 3D models into a compact triplane latent space to effectively compress both the 3D geometry and texture information. Within the autoencoder framework, we introduce a 3D-aware cross-attention mechanism, which utilizes low-resolution latent representations to query features from a high-resolution 3D feature volume, thereby enhancing the representation capacity of the latent space. Subsequently, we train a diffusion model on this refined latent space. In contrast to solely relying on image embedding for 3D generation, our proposed method advocates for the simultaneous utilization of both image embedding and shape embedding as conditions. Specifically, the shape embedding is estimated via a diffusion prior model conditioned on the image embedding. Through comprehensive experiments, we demonstrate that our method outperforms state-of-the-art algorithms, achieving superior performance while requiring less training data and time. Our approach enables the generation of high-quality 3D assets in merely 7 seconds on a single A100 GPU.

翻译：三维生成技术已取得显著进展，但从单张图像高效生成高质量三维资产仍具挑战性。本文提出了一种三平面自编码器，通过将三维模型编码至紧凑的三平面潜在空间，有效压缩三维几何与纹理信息。在该自编码器框架中，我们引入三维感知交叉注意力机制，利用低分辨率潜在表征从高分辨率三维特征体中查询特征，从而增强潜在空间的表征能力。随后，我们在优化后的潜在空间上训练扩散模型。不同于仅依赖图像嵌入进行三维生成，本方法主张同时利用图像嵌入与形状嵌入作为条件，其中形状嵌入通过基于图像嵌入的扩散先验模型进行估计。综合实验表明，本方法在减少训练数据与时间的前提下，性能优于现有最优算法。我们的方法可在单张A100 GPU上仅需7秒生成高质量三维资产。