Physical experiments often involve multiple imaging representations, such as X-ray scans and microscopic images. Deep learning models have been widely used for supervised analysis in these experiments. Combining different image representations is frequently required to analyze and make a decision properly. Consequently, multi-view data has emerged - datasets where each sample is described by views from different angles, sources, or modalities. These problems are addressed with the concept of multi-view learning. Understanding the decision-making process of deep learning models is essential for reliable and credible analysis. Hence, many explainability methods have been devised recently. Nonetheless, there is a lack of proper explainability in multi-view models, which are challenging to explain due to their architectures. In this paper, we suggest different multi-view architectures for the vision domain, each suited to another problem, and we also present a methodology for explaining these models. To demonstrate the effectiveness of our methodology, we focus on the domain of High Energy Density Physics (HEDP) experiments, where multiple imaging representations are used to assess the quality of foam samples. We apply our methodology to classify the foam samples quality using the suggested multi-view architectures. Through experimental results, we showcase the improvement of accurate architecture choice on both accuracy - 78% to 84% and AUC - 83% to 93% and present a trade-off between performance and explainability. Specifically, we demonstrate that our approach enables the explanation of individual one-view models, providing insights into the decision-making process of each view. This understanding enhances the interpretability of the overall multi-view model. The sources of this work are available at: https://github.com/Scientific-Computing-Lab-NRCN/Multi-View-Explainability.

翻译：物理实验通常涉及多种成像表征方式，例如X射线扫描和显微图像。深度学习模型已广泛应用于这些实验中的监督分析。为正确分析和决策，常需结合不同图像表征。由此产生了多视角数据——即每个样本通过不同角度、来源或模态的视角进行描述的数据集。这类问题可通过多视角学习的概念加以解决。理解深度学习模型的决策过程对实现可靠可信的分析至关重要，因此近年涌现出众多可解释性方法。然而，由于多视角模型架构的复杂性，其缺乏适当的可解释性。本文针对视觉领域提出多种多视角架构，每种架构适用于不同问题，并同时提出解释这些模型的方法论。为证明方法论的有效性，我们聚焦于高能量密度物理（HEDP）实验领域——该领域通过多种成像表征评估泡沫样本质量。我们应用所提方法论，利用建议的多视角架构对泡沫样本质量进行分类。通过实验结果，我们展示了精确架构选择带来的准确率提升（从78%提高至84%）以及AUC性能提升（从83%提高至93%），并揭示了性能与可解释性之间的权衡关系。特别地，我们证明该方法能够解释单个视角模型，为每个视角的决策过程提供洞察。这种理解增强了整体多视角模型的可解释性。本工作源代码见：https://github.com/Scientific-Computing-Lab-NRCN/Multi-View-Explainability