Instruction tuning large language models (LLMs) remains a challenging task, owing to the complexity of hyperparameter selection and the difficulty involved in evaluating the tuned models. To determine the optimal hyperparameters, an automatic, robust, and reliable evaluation benchmark is essential. However, establishing such a benchmark is not a trivial task due to the challenges associated with evaluation accuracy and privacy protection. In response to these challenges, we introduce a judge large language model, named PandaLM, which is trained to distinguish the superior model given several LLMs. PandaLM's focus extends beyond just the objective correctness of responses, which is the main focus of traditional evaluation datasets. It addresses vital subjective factors such as relative conciseness, clarity, adherence to instructions, comprehensiveness, and formality. To ensure the reliability of PandaLM, we collect a diverse human-annotated test dataset, where all contexts are generated by humans and labels are aligned with human preferences. Our results indicate that PandaLM-7B achieves 93.75% of GPT-3.5's evaluation ability and 88.28% of GPT-4's in terms of F1-score on our test dataset. PandaLM enables the evaluation of LLM to be fairer but with less cost, evidenced by significant improvements achieved by models tuned through PandaLM compared to their counterparts trained with default Alpaca's hyperparameters. In addition, PandaLM does not depend on API-based evaluations, thus avoiding potential data leakage. All resources of PandaLM are released at https://github.com/WeOpenML/PandaLM.

翻译：指令微调大语言模型仍是一项具有挑战性的任务，这源于超参数选择的复杂性以及评估微调模型本身的困难。为确定最优超参数，自动、稳健且可靠的评估基准至关重要。然而，由于评估准确性和隐私保护方面的挑战，构建这样的基准并非易事。针对这些问题，我们引入了一个名为PandaLM的评判型大语言模型，该模型经过训练能够从多个大语言模型中区分出性能更优的模型。PandaLM的关注点不仅局限于传统评估数据集主要关注的目标答案客观正确性，还涵盖了相对简洁性、清晰度、指令遵循程度、全面性和正式性等关键主观因素。为确保PandaLM的可靠性，我们收集了多样化的人工标注测试数据集，其中所有上下文均由人工生成，标签与人类偏好保持一致。实验结果表明，在我们的测试数据集上，PandaLM-7B在F1分数上达到了GPT-3.5评估能力的93.75%和GPT-4的88.28%。PandaLM使得大语言模型评估更加公平且成本更低，这通过使用PandaLM微调的模型相比采用默认Alpaca超参数训练的模型所取得的显著改进得以验证。此外，PandaLM不依赖基于API的评估，从而避免了潜在的数据泄露。PandaLM的所有资源已发布于https://github.com/WeOpenML/PandaLM。