The current study focuses on systematically analyzing the recent advances in the field of Multimodal eXplainable Artificial Intelligence (MXAI). In particular, the relevant primary prediction tasks and publicly available datasets are initially described. Subsequently, a structured presentation of the MXAI methods of the literature is provided, taking into account the following criteria: a) The number of the involved modalities, b) The stage at which explanations are produced, and c) The type of the adopted methodology (i.e. mathematical formalism). Then, the metrics used for MXAI evaluation are discussed. Finally, a comprehensive analysis of current challenges and future research directions is provided.
翻译:本研究旨在系统分析多模态可解释人工智能领域的最新进展。首先,详细介绍相关的主要预测任务与公开可用的数据集。继而,基于以下标准对文献中的多模态可解释人工智能方法进行结构化呈现:a)所涉及模态的数量,b)解释生成的阶段,以及c)所采用方法论的类型(即数学形式化)。随后,讨论用于多模态可解释人工智能评估的度量指标。最后,全面分析当前面临的挑战与未来研究方向。