The deployment of multimodal models in high-stakes domains, such as self-driving vehicles and medical diagnostics, demands not only strong predictive performance but also reliable mechanisms for detecting failures. In this work, we address the largely unexplored problem of failure detection in multimodal contexts. We propose Adaptive Confidence Regularization (ACR), a novel framework specifically designed to detect multimodal failures. Our approach is driven by a key observation: in most failure cases, the confidence of the multimodal prediction is significantly lower than that of at least one unimodal branch, a phenomenon we term confidence degradation. To mitigate this, we introduce an Adaptive Confidence Loss that penalizes such degradations during training. In addition, we propose Multimodal Feature Swapping, a novel outlier synthesis technique that generates challenging, failure-aware training examples. By training with these synthetic failures, ACR learns to more effectively recognize and reject uncertain predictions, thereby improving overall reliability. Extensive experiments across four datasets, three modalities, and multiple evaluation settings demonstrate that ACR achieves consistent and robust gains. The source code will be available at https://github.com/mona4399/ACR.

翻译：在高风险领域（如自动驾驶车辆和医疗诊断）部署多模态模型时，不仅需要强大的预测性能，还需要可靠的故障检测机制。本研究针对多模态场景中尚未充分探索的故障检测问题，提出了自适应置信度正则化（ACR）这一专门用于检测多模态故障的新框架。我们的方法基于一个关键观察：在大多数故障情况下，多模态预测的置信度显著低于至少一个单模态分支的置信度，我们将此现象称为置信度退化。为缓解此问题，我们引入了自适应置信度损失函数，在训练过程中对此类退化进行惩罚。此外，我们提出了多模态特征交换技术，这是一种新颖的异常值合成方法，能够生成具有挑战性且包含故障信息的训练样本。通过使用这些合成故障进行训练，ACR能够更有效地识别并拒绝不确定的预测，从而提升整体可靠性。我们在四个数据集、三种模态及多种评估设置下进行了广泛实验，结果表明ACR取得了持续且稳健的性能提升。源代码将在 https://github.com/mona4399/ACR 公开。