This work explores multi-modal inference in a high-dimensional simplified model, analytically quantifying the performance gain of multi-modal inference over that of analyzing modalities in isolation. We present the Bayes-optimal performance and weak recovery thresholds in a model where the objective is to recover the latent structures from two noisy data matrices with correlated spikes. The paper derives the approximate message passing (AMP) algorithm for this model and characterizes its performance in the high-dimensional limit via the associated state evolution. The analysis holds for a broad range of priors and noise channels, which can differ across modalities. The linearization of AMP is compared numerically to the widely used partial least squares (PLS) and canonical correlation analysis (CCA) methods, which are both observed to suffer from a sub-optimal recovery threshold.

翻译：本研究探索了一个高维简化模型中的多模态推断问题，从理论上量化了多模态推断相较于单模态分析的性能增益。我们提出了一个模型中的贝叶斯最优性能与弱恢复阈值，该模型的目标是从两个具有相关尖峰信号的含噪数据矩阵中恢复潜在结构。本文推导了适用于该模型的近似消息传递（AMP）算法，并通过关联的状态演化方程刻画了其在维数趋于无穷时的性能。该分析适用于广泛的先验分布与噪声信道，且各模态间可存在差异。通过数值实验将AMP的线性化版本与广泛使用的偏最小二乘（PLS）和典型相关分析（CCA）方法进行比较，发现后两者均存在恢复阈值次优的问题。