Handling implicit language is essential for natural language processing systems to achieve precise text understanding and facilitate natural interactions with users. Despite its importance, the absence of a robust metric for accurately measuring the implicitness of language significantly constrains the depth of analysis possible in evaluating models' comprehension capabilities. This paper addresses this gap by developing a scalar metric that quantifies the implicitness level of language without relying on external references. Drawing on principles from traditional linguistics, we define ''implicitness'' as the divergence between semantic meaning and pragmatic interpretation. To operationalize this definition, we introduce ImpScore, a novel, reference-free metric formulated through an interpretable regression model. This model is trained using pairwise contrastive learning on a specially curated dataset comprising $112,580$ (implicit sentence, explicit sentence) pairs. We validate ImpScore through a user study that compares its assessments with human evaluations on out-of-distribution data, demonstrating its accuracy and strong correlation with human judgments. Additionally, we apply ImpScore to hate speech detection datasets, illustrating its utility and highlighting significant limitations in current large language models' ability to understand highly implicit content. The metric model and its training data are available at https://github.com/audreycs/ImpScore.

翻译：处理隐含语言对于自然语言处理系统实现精确文本理解并促进与用户的自然交互至关重要。尽管其重要性不言而喻，但由于缺乏能够准确衡量语言隐含程度的稳健度量指标，极大地限制了对模型理解能力进行深入分析的深度。本文通过开发一种不依赖外部参考、能够量化语言隐含程度的标量度量指标，以弥补这一空白。借鉴传统语言学原理，我们将“隐含性”定义为语义含义与语用解读之间的差异。为实现这一定义的可操作化，我们引入了ImpScore——一种通过可解释回归模型构建的新型无参考度量指标。该模型使用专门构建的数据集（包含$112,580$对（隐含句，显性句）），通过成对对比学习进行训练。我们通过一项用户研究验证了ImpScore的有效性，该研究将其在分布外数据上的评估结果与人工评估进行对比，证明了其准确性以及与人类判断的强相关性。此外，我们将ImpScore应用于仇恨言论检测数据集，展示了其实用性，并凸显了当前大语言模型在理解高度隐含内容方面存在的显著局限性。该度量模型及其训练数据可在 https://github.com/audreycs/ImpScore 获取。