Multimodal datasets contain observations generated by multiple types of sensors. Most works to date focus on uncovering latent structures in the data that appear in all modalities. However, important aspects of the data may appear in only one modality due to the differences between the sensors. Uncovering modality-specific attributes may provide insights into the sources of the variability of the data. For example, certain clusters may appear in the analysis of genetics but not in epigenetic markers. Another example is hyper-spectral satellite imaging, where various atmospheric and ground phenomena are detectable using different parts of the spectrum. In this paper, we address the problem of uncovering latent structures that are unique to a single modality. Our approach is based on computing a graph representation of datasets from two modalities and analyzing the differences between their connectivity patterns. We provide an asymptotic analysis of the convergence of our approach based on a product manifold model. To evaluate the performance of our method, we test its ability to uncover latent structures in multiple types of artificial and real datasets.

翻译：多模态数据集包含由多种类型传感器生成的观测结果。迄今为止，大多数研究集中于揭示所有模态中均出现的数据潜在结构。然而，由于传感器间的差异，数据的重要方面可能仅出现在单一模态中。揭示模态特异性属性可为数据变异性的来源提供洞见。例如，某些聚类可能在遗传学分析中出现，但在表观遗传标记中不出现。另一示例是高光谱卫星成像，其中不同的大气和地面现象可通过光谱的不同部分进行检测。本文致力于解决揭示单一模态特有潜在结构的问题。我们的方法基于计算来自两个模态的数据集图表示，并分析其连接模式间的差异。我们基于乘积流形模型提供了该方法收敛性的渐近分析。为评估本方法的性能，我们在多种人工和真实数据集上测试其揭示潜在结构的能力。