Prompt optimization aims to systematically refine prompts to enhance a language model's performance on specific tasks. Fairness detection in Terms of Service (ToS) clauses is a challenging legal NLP task that demands carefully crafted prompts to ensure reliable results. However, existing prompt optimization methods are often computationally expensive due to inefficient search strategies and costly prompt candidate scoring. In this paper, we propose a framework that combines Monte Carlo Tree Search (MCTS) with a proxy prompt evaluator to more effectively explore the prompt space while reducing evaluation costs. Experiments demonstrate that our approach achieves higher classification accuracy and efficiency than baseline methods under a constrained computation budget.
翻译:提示优化的目标是通过系统性地改进提示来提升语言模型在特定任务上的性能。服务条款(ToS)中的公平性检测是一项具有挑战性的法律自然语言处理任务,需要精心设计的提示以确保可靠的结果。然而,现有的提示优化方法由于搜索策略效率低下和提示候选评分成本高昂,往往计算开销巨大。本文提出一种结合蒙特卡洛树搜索(MCTS)与代理提示评估器的框架,以更有效地探索提示空间,同时降低评估成本。实验表明,在有限的计算预算下,我们的方法相比基线方法实现了更高的分类准确率与效率。