This research paper proposes a Latent Diffusion Model for 3D (LDM3D) that generates both image and depth map data from a given text prompt, allowing users to generate RGBD images from text prompts. The LDM3D model is fine-tuned on a dataset of tuples containing an RGB image, depth map and caption, and validated through extensive experiments. We also develop an application called DepthFusion, which uses the generated RGB images and depth maps to create immersive and interactive 360-degree-view experiences using TouchDesigner. This technology has the potential to transform a wide range of industries, from entertainment and gaming to architecture and design. Overall, this paper presents a significant contribution to the field of generative AI and computer vision, and showcases the potential of LDM3D and DepthFusion to revolutionize content creation and digital experiences. A short video summarizing the approach can be found at https://t.ly/tdi2.

翻译：本研究提出了一种面向3D的潜在扩散模型（LDM3D），该模型能够根据给定的文本提示生成图像和深度图数据，使用户能够从文本提示生成RGBD图像。LDM3D模型在包含RGB图像、深度图和标题的三元组数据集上进行了微调，并通过大量实验进行了验证。我们还开发了一个名为DepthFusion的应用程序，该应用程序利用生成的RGB图像和深度图，通过TouchDesigner创建沉浸式且可交互的360度视角体验。这项技术有潜力彻底改变从娱乐、游戏到建筑和设计等广泛行业。总的来说，本文为生成式人工智能和计算机视觉领域做出了重要贡献，并展示了LDM3D和DepthFusion在革新内容创作和数字体验方面的潜力。一段总结该方法的短视频可访问 https://t.ly/tdi2。