Multimodal datasets contain observations generated by multiple types of sensors. Most works to date focus on uncovering latent structures in the data that appear in all modalities. However, important aspects of the data may appear in only one modality due to the differences between the sensors. Uncovering modality-specific attributes may provide insights into the sources of the variability of the data. For example, certain clusters may appear in the analysis of genetics but not in epigenetic markers. Another example is hyper-spectral satellite imaging, where various atmospheric and ground phenomena are detectable using different parts of the spectrum. In this paper, we address the problem of uncovering latent structures that are unique to a single modality. Our approach is based on computing a graph representation of datasets from two modalities and analyzing the differences between their connectivity patterns. We provide an asymptotic analysis of the convergence of our approach based on a product manifold model. To evaluate the performance of our method, we test its ability to uncover latent structures in multiple types of artificial and real datasets.

翻译：多模态数据集包含由多种类型传感器生成的观测数据。现有研究大多聚焦于发现所有模态中共同出现的潜在数据结构。然而，由于传感器差异，数据的重要方面可能仅出现在单一模态中。揭示模态特异性属性有助于洞察数据变异性的来源。例如，某些聚类可能出现在遗传学分析中，但在表观遗传标记中并不显现。另一个例子是高光谱卫星成像，其中不同大气和地面现象可通过光谱的不同部分进行检测。本文探讨了揭示单一模态中独有潜在结构的问题。我们的方法基于计算两种模态数据集的图表示，并分析其连通模式的差异。我们基于乘积流形模型对所提方法的收敛性进行了渐近分析。为评估方法性能，我们测试了其在多种人工数据集与真实数据集中揭示潜在结构的能力。