During multimodal model training and testing, certain data modalities may be absent due to sensor limitations, cost constraints, privacy concerns, or data loss, negatively affecting performance. Multimodal learning techniques designed to handle missing modalities can mitigate this by ensuring model robustness even when some modalities are unavailable. This survey reviews recent progress in Multimodal Learning with Missing Modality (MLMM), focusing on deep learning methods. It provides the first comprehensive survey that covers the motivation and distinctions between MLMM and standard multimodal learning setups, followed by a detailed analysis of current methods, applications, and datasets, concluding with challenges and future directions.
翻译:在多模态模型的训练与测试过程中,由于传感器限制、成本约束、隐私考量或数据丢失等原因,某些数据模态可能出现缺失,从而对模型性能产生负面影响。专门为处理缺失模态而设计的多模态学习技术能够缓解这一问题,确保模型在部分模态不可用时仍保持鲁棒性。本综述回顾了缺失模态多模态学习(MLMM)领域的最新进展,重点关注深度学习方法。本文首次提供了全面综述,涵盖MLMM与标准多模态学习设置的动机与区别,随后对现有方法、应用场景及数据集进行了详细分析,最后总结了当前面临的挑战与未来研究方向。