Multimodal Large Language Models (MLLMs) exhibit promising advancements across various tasks, yet they still encounter significant trustworthiness issues. Prior studies apply Split Conformal Prediction (SCP) in language modeling to construct prediction sets with statistical guarantees. However, these methods typically rely on internal model logits or are restricted to multiple-choice settings, which hampers their generalizability and adaptability in dynamic, open-ended environments. In this paper, we introduce TRON, a two-step framework for risk control and assessment, applicable to any MLLM that supports sampling in both open-ended and closed-ended scenarios. TRON comprises two main components: (1) a novel conformal score to sample response sets of minimum size, and (2) a nonconformity score to identify high-quality responses based on self-consistency theory, controlling the error rates by two specific risk levels. Furthermore, we investigate semantic redundancy in prediction sets within open-ended contexts for the first time, leading to a promising evaluation metric for MLLMs based on average set size. Our comprehensive experiments across four Video Question-Answering (VideoQA) datasets utilizing eight MLLMs show that TRON achieves desired error rates bounded by two user-specified risk levels. Additionally, deduplicated prediction sets maintain adaptiveness while being more efficient and stable for risk assessment under different risk levels.

翻译：多模态大语言模型（MLLMs）在各种任务中展现出前景广阔的进展，但仍面临显著的可信度问题。先前研究将分割保形预测（SCP）应用于语言建模，以构建具有统计保证的预测集。然而，这些方法通常依赖于模型内部逻辑值或局限于多项选择设置，这限制了其在动态、开放式环境中的泛化能力和适应性。本文提出TRON，一个适用于支持开放式和封闭式场景采样的任意MLLM的两步风险控制与评估框架。TRON包含两个核心组件：（1）一种新颖的保形评分方法，用于采样最小规模的响应集；（2）基于自洽理论设计的不保和度评分方法，通过两个特定风险水平识别高质量响应并控制错误率。此外，我们首次研究了开放式语境下预测集中的语义冗余问题，由此提出一种基于平均集大小的、有前景的MLLM评估指标。我们在四个视频问答（VideoQA）数据集上使用八个MLLM进行的全面实验表明，TRON能够实现由用户指定的两个风险水平界定的目标错误率。此外，去重后的预测集在不同风险水平下进行风险评估时，在保持适应性的同时展现出更高的效率与稳定性。