Since the natural language processing (NLP) community started to make large language models (LLMs), such as GPT-4, act as a critic to evaluate the quality of generated texts, most of them only train a critique generation model of a specific scale on specific datasets. We argue that a comprehensive investigation on the key factor of LLM-based evaluation models, such as scaling properties, is lacking, so that it is still inconclusive whether these models have potential to replace GPT-4's evaluation in practical scenarios. In this paper, we propose a new critique generation model called CritiqueLLM, which includes a dialogue-based prompting method for high-quality referenced / reference-free evaluation data. Experimental results show that our model can achieve comparable evaluation performance to GPT-4 especially in system-level correlations, and even outperform GPT-4 in 3 out of 8 tasks in a challenging reference-free setting. We conduct detailed analysis to show promising scaling properties of our model in the quality of generated critiques. We also demonstrate that our generated critiques can act as scalable feedback to directly improve the generation quality of LLMs.

翻译：自自然语言处理领域开始使用GPT-4等大型语言模型作为批评者评估生成文本质量以来，大多数研究仅针对特定数据集训练特定规模的批评生成模型。我们认为，当前缺乏对基于LLM的评估模型关键因素（如缩放特性）的系统性研究，因此尚无法确定这些模型能否替代GPT-4在实际场景中的评估能力。本文提出一种新型批评生成模型CritiqueLLM，其采用基于对话的提示方法生成高质量有参考/无参考评估数据。实验结果表明，我们的模型在系统级相关性等指标上能达到与GPT-4相当的评估性能，甚至在极具挑战性的无参考设置下，8个任务中有3个任务的表现超越了GPT-4。通过详尽分析，我们展示了所生成批评质量具有良好缩放特性。此外，我们还证明了生成的批评可作为可扩展的反馈信号，直接提升大型语言模型的生成质量。