Estimating question difficulty is a critical component in evaluating and improving large language models (LLMs) for question answering (QA). Existing approaches often rely on readability formulas, retrieval-based signals, or popularity statistics, which may not fully capture the reasoning challenges posed to modern LLMs. In this paper, we introduce Q-DAPS (Question Difficulty based on Answer Plausibility Scores) method, a novel approach that estimates question difficulty by computing the entropy of plausibility scores over candidate answers. We systematically evaluate Q-DAPS across four prominent QA datasets-TriviaQA, NQ, MuSiQue, and QASC-demonstrating that it consistently outperforms baselines. Moreover, Q-DAPS shows strong robustness across hyperparameter variations and question types. Extensive ablation studies further show that Q-DAPS remains robust across different plausibility estimation paradigms, model sizes, and realistic settings. Human evaluations further confirm strong alignment between Q-DAPS's difficulty estimates and human judgments of question difficulty. Overall, Q-DAPS provides an interpretable, scalable, and bias-resilient approach to question difficulty estimation in modern QA systems.

翻译：问题难度估计是评估和改进大语言模型（LLMs）问答能力的关键组成部分。现有方法通常依赖可读性公式、检索信号或流行度统计，但这些指标可能无法完全捕捉现代LLM所面临的推理挑战。本文提出Q-DAPS（基于答案合理性问题难度评分）方法，通过计算候选答案合理性评分的熵值来估计问题难度。我们在四个主流问答数据集（TriviaQA、NQ、MuSiQue和QASC）上系统评估了Q-DAPS，证明其性能始终优于基线方法。此外，Q-DAPS在超参数变化和不同问题类型上表现出强大的鲁棒性。大量消融实验进一步表明，在不同合理性估计范式、模型规模及现实场景下，Q-DAPS均保持稳健。人类评估也证实Q-DAPS的难度估计与人类判断高度一致。总体而言，Q-DAPS为现代问答系统中的问题难度估计提供了一种可解释、可扩展且具有抗偏性的解决方案。