3D image reconstruction from a limited number of 2D images has been a long-standing challenge in computer vision and image analysis. While deep learning-based approaches have achieved impressive performance in this area, existing deep networks often fail to effectively utilize the shape structures of objects presented in images. As a result, the topology of reconstructed objects may not be well preserved, leading to the presence of artifacts such as discontinuities, holes, or mismatched connections between different parts. In this paper, we propose a shape-aware network based on diffusion models for 3D image reconstruction, named SADIR, to address these issues. In contrast to previous methods that primarily rely on spatial correlations of image intensities for 3D reconstruction, our model leverages shape priors learned from the training data to guide the reconstruction process. To achieve this, we develop a joint learning network that simultaneously learns a mean shape under deformation models. Each reconstructed image is then considered as a deformed variant of the mean shape. We validate our model, SADIR, on both brain and cardiac magnetic resonance images (MRIs). Experimental results show that our method outperforms the baselines with lower reconstruction error and better preservation of the shape structure of objects within the images.

翻译：从有限数量的二维图像进行三维重建一直是计算机视觉与图像分析领域的长期挑战。尽管基于深度学习的相关方法已取得显著成效，但现有深度网络往往难以有效利用图像中物体的形状结构。这可能导致重建物体的拓扑结构无法得到良好保持，从而产生不连续、空洞或不同部件间连接错位等伪影。针对上述问题，本文提出一种基于扩散模型的形状感知网络SADIR，用于三维图像重建。与以往主要依赖图像强度空间相关性进行三维重建的方法不同，本模型利用从训练数据中学习到的形状先验来引导重建过程。为此，我们开发了一个联合学习网络，该网络能够同时学习变形模型下的平均形状，并将每个重建图像视为平均形状的变形变体。我们分别在脑部与心脏磁共振图像上验证了SADIR模型的性能。实验结果表明，本方法在降低重建误差、更好保持图像中物体的形状结构方面均优于基线模型。