In recent years, substantial advancements have been made in the development of large language models, achieving remarkable performance across diverse tasks. To evaluate the knowledge ability of language models, previous studies have proposed lots of benchmarks based on question-answering pairs. We argue that it is not reliable and comprehensive to evaluate language models with a fixed question or limited paraphrases as the query, since language models are sensitive to prompt. Therefore, we introduce a novel concept named knowledge boundary to encompass both prompt-agnostic and prompt-sensitive knowledge within language models. Knowledge boundary avoids prompt sensitivity in language model evaluations, rendering them more dependable and robust. To explore the knowledge boundary for a given model, we propose projected gradient descent method with semantic constraints, a new algorithm designed to identify the optimal prompt for each piece of knowledge. Experiments demonstrate a superior performance of our algorithm in computing the knowledge boundary compared to existing methods. Furthermore, we evaluate the ability of multiple language models in several domains with knowledge boundary.

翻译：近年来，大型语言模型的发展取得了实质性进展，在多样化任务中展现出卓越性能。为评估语言模型的知识能力，先前研究已提出众多基于问答对的基准测试。我们认为，使用固定问题或有限释义作为查询来评估语言模型并不可靠且不够全面，因为语言模型对提示词具有敏感性。为此，我们引入名为"知识边界"的新概念，用以涵盖语言模型中既存的提示无关知识与提示敏感知识。知识边界避免了语言模型评估中的提示敏感性，使评估结果更具可靠性与鲁棒性。为探索给定模型的知识边界，我们提出具有语义约束的投影梯度下降法——这是一种旨在为每条知识寻找最优提示词的新算法。实验证明，在计算知识边界方面，本算法较现有方法具有更优性能。此外，我们运用知识边界对多个语言模型在若干领域的能力进行了评估。