We present a formal problem formulation for \textit{Reliable} Audio-Visual Question Answering ($\mathcal{R}$-AVQA), where we prefer abstention over answering incorrectly. While recent AVQA models have high accuracy, their ability to identify when they are likely wrong and their consequent abstention from answering remain underexplored areas of research. To fill this gap, we explore several approaches and then propose Adaptive Confidence Refinement (ACR), a lightweight method to further enhance the performance of $\mathcal{R}$-AVQA. Our key insight is that the Maximum Softmax Probability (MSP) is Bayes-optimal only under strong calibration, a condition usually not met in deep neural networks, particularly in multimodal models. Instead of replacing MSP, our ACR maintains it as a primary confidence signal and applies input-adaptive residual corrections when MSP is deemed unreliable. ACR introduces two learned heads: i) a Residual Risk Head that predicts low-magnitude correctness residuals that MSP does not capture, and ii) a Confidence Gating Head to determine MSP trustworthiness. Our experiments and theoretical analysis show that ACR consistently outperforms existing methods on in- and out-of-disrtibution, and data bias settings across three different AVQA architectures, establishing a solid foundation for $\mathcal{R}$-AVQA task. The code and checkpoints will be available upon acceptance \href{https://github.com/PhuTran1005/R-AVQA}{at here}

翻译：本文针对**可靠**视听问答（$\mathcal{R}$-AVQA）提出了一个形式化的问题定义，其核心原则是：当模型不确定时，宁可选择弃答，也不愿给出错误答案。尽管当前的AVQA模型已具备较高的准确率，但其识别自身潜在错误并据此主动弃答的能力，仍是研究中尚未充分探索的领域。为填补这一空白，我们探索了多种方法，进而提出了自适应置信度优化（ACR）——一种轻量级方法，旨在进一步提升$\mathcal{R}$-AVQA的性能。我们的核心见解是：最大软最大概率（MSP）仅在强校准条件下才是贝叶斯最优的，而这一条件在深度神经网络中通常难以满足，尤其是在多模态模型中。ACR并未取代MSP，而是将其保留为主要置信度信号，并在判定MSP不可靠时，施加输入自适应的残差校正。ACR引入了两个可学习的头部：i) **残差风险头部**，用于预测MSP未能捕捉到的低幅度正确性残差；ii) **置信度门控头部**，用于判定MSP的可信度。我们的实验与理论分析表明，在三种不同的AVQA架构上，ACR在分布内、分布外以及存在数据偏差的设置中，均持续优于现有方法，从而为$\mathcal{R}$-AVQA任务奠定了坚实基础。代码与检查点将在论文被接受后于\href{https://github.com/PhuTran1005/R-AVQA}{此处}提供。