Intraoperative shape reconstruction of organs from endoscopic camera images is a complex yet indispensable technique for image-guided surgery. To address the uncertainty in reconstructing entire shapes from single-viewpoint occluded images, we propose a framework for generative virtual learning of shape reconstruction using image translation with common latent variables between simulated and real images. As it is difficult to prepare sufficient amount of data to learn the relationship between endoscopic images and organ shapes, self-supervised virtual learning is performed using simulated images generated from statistical shape models. However, small differences between virtual and real images can degrade the estimation performance even if the simulated images are regarded as equivalent by humans. To address this issue, a Variational Autoencoder is used to convert real and simulated images into identical synthetic images. In this study, we targeted the shape reconstruction of collapsed lungs from thoracoscopic images and confirmed that virtual learning could improve the similarity between real and simulated images. Furthermore, shape reconstruction error could be improved by 16.9%.

翻译：术中从内窥镜相机图像中重建器官形状是图像引导手术中复杂且必不可少的技术。为了解决从单视角遮挡图像重建完整形状的不确定性，我们提出了一种生成式虚拟学习框架，该框架利用模拟图像与真实图像之间的共享潜在变量进行图像翻译来实现形状重建。由于难以准备足够的数据来学习内窥镜图像与器官形状之间的关系，我们使用从统计形状模型生成的模拟图像进行自监督虚拟学习。然而，即使模拟图像在人类看来与真实图像等同，两者之间的微小差异也可能降低估计性能。为解决这一问题，我们采用变分自编码器将真实图像与模拟图像转换为相同的合成图像。本研究以胸腔镜图像中塌陷肺的形状重建为目标，证实了虚拟学习能够提高真实图像与模拟图像之间的相似性。此外，形状重建误差降低了16.9%。