There is a significant interest in exploring non-linear associations among multiple images derived from diverse imaging modalities. While there is a growing literature on image-on-image regression to delineate predictive inference of an image based on multiple images, existing approaches have limitations in efficiently borrowing information between multiple imaging modalities in the prediction of an image. Building on the literature of Variational Auto Encoders (VAEs), this article proposes a novel approach, referred to as Integrative Variational Autoencoder (\texttt{InVA}) method, which borrows information from multiple images obtained from different sources to draw predictive inference of an image. The proposed approach captures complex non-linear association between the outcome image and input images, while allowing rapid computation. Numerical results demonstrate substantial advantages of \texttt{InVA} over VAEs, which typically do not allow borrowing information between input images. The proposed framework offers highly accurate predictive inferences for costly positron emission topography (PET) from multiple measures of cortical structure in human brain scans readily available from magnetic resonance imaging (MRI).

翻译：摘要：探索来自不同成像模态的多幅图像之间的非线性关联具有重要研究价值。尽管现有文献中关于图像对图像回归（用以基于多幅图像描述预测性推断）的研究日益增多，但现有方法在有效融合多模态影像信息进行图像预测方面仍存在局限性。本文基于变分自编码器（VAEs）文献，提出一种名为积分变分自编码器（\texttt{InVA}）的新方法，该方法能够整合来自不同来源的多幅图像信息，从而实现对目标图像的预测性推断。所提方法不仅捕捉结果图像与输入图像之间的复杂非线性关联，同时保持快速计算能力。数值结果表明，\texttt{InVA}相较于通常无法整合输入图像间信息的标准VAE具有显著优势。该框架可利用磁共振成像（MRI）获取的皮质结构多项测量指标，对成本高昂的正电子发射断层扫描（PET）实现高精度预测性推断。