Large language models (LLMs) have shown promising abilities as cost-effective and reference-free evaluators for assessing language generation quality. In particular, pairwise LLM evaluators, which compare two generated texts and determine the preferred one, have been employed in a wide range of applications. However, LLMs exhibit preference biases and worrying sensitivity to prompt designs. In this work, we first reveal that the predictive preference of LLMs can be highly brittle and skewed, even with semantically equivalent instructions. We find that fairer predictive preferences from LLMs consistently lead to judgments that are better aligned with humans. Motivated by this phenomenon, we propose an automatic Zero-shot Evaluation-oriented Prompt Optimization framework, ZEPO, which aims to produce fairer preference decisions and improve the alignment of LLM evaluators with human judgments. To this end, we propose a zero-shot learning objective based on the preference decision fairness. ZEPO demonstrates substantial performance improvements over state-of-the-art LLM evaluators, without requiring labeled data, on representative meta-evaluation benchmarks. Our findings underscore the critical correlation between preference fairness and human alignment, positioning ZEPO as an efficient prompt optimizer for bridging the gap between LLM evaluators and human judgments.

翻译：大语言模型（LLMs）已展现出作为评估语言生成质量的经济高效且无需参考标准的评估器的潜力。特别是成对LLM评估器，通过比较两个生成文本并确定更优者，已在广泛的应用中得到使用。然而，LLMs表现出偏好偏见以及对提示设计令人担忧的敏感性。在本工作中，我们首先揭示了LLMs的预测偏好可能极其脆弱且存在偏差，即使在使用语义等价的指令时也是如此。我们发现，来自LLMs的更公平的预测偏好能够持续产生与人类判断更佳对齐的评判。受此现象启发，我们提出了一种自动化的零样本评估导向提示优化框架——ZEPO，其旨在产生更公平的偏好决策，并提升LLM评估器与人类判断的对齐度。为此，我们提出了一种基于偏好决策公平性的零样本学习目标。在具有代表性的元评估基准测试中，ZEPO相较于最先进的LLM评估器展现出显著的性能提升，且无需标注数据。我们的研究结果强调了偏好公平性与人类对齐之间的关键关联，并将ZEPO定位为一种高效的提示优化器，用于弥合LLM评估器与人类判断之间的差距。