Bargaining is an important and unique part of negotiation between humans. As LLM-driven agents learn to negotiate and act like real humans, how to evaluate agents' bargaining abilities remains an open problem. For the first time, we formally described the Bargaining task as an asymmetric incomplete information game, defining the gains of the Buyer and Seller in multiple bargaining processes. It allows us to quantitatively assess an agent's performance in the Bargain task. We collected a real product price dataset, AmazonHistoryPrice, and conducted evaluations of various LLM agents' bargaining abilities. We find that playing a Buyer is much harder than a Seller, and increasing model size can not effectively improve the Buyer's performance. To address the challenge, we propose a novel approach called OG-Narrator that integrates a deterministic Offer Generator to control the price range of Buyer's offers, and an LLM Narrator to create natural language sentences for generated offers. Experimental results show that OG-Narrator improves the buyer's deal rates from 26.67% to 88.88% and brings a ten times of multiplication of profits on all baselines, even a model that has not been aligned.

翻译：议价是人类谈判中重要且独特的一部分。随着基于大语言模型的智能体学习像真实人类一样进行谈判和行动，如何评估这些智能体的议价能力仍是一个待解决的问题。我们首次将议价任务形式化为一种非对称不完全信息博弈，定义了买方和卖方在多个议价过程中的收益。这使得我们能够定量评估智能体在议价任务中的表现。我们收集了一个真实商品价格数据集AmazonHistoryPrice，并对多种大语言模型智能体的议价能力进行了评估。研究发现，扮演买方远比扮演卖方困难，且增大模型规模并不能有效提升买方的表现。为解决这一挑战，我们提出了一种名为OG-Narrator的新方法，它集成了一个确定性报价生成器以控制买方报价的价格范围，以及一个大语言模型叙述器为生成的报价创建自然语言句子。实验结果表明，OG-Narrator将买方的成交率从26.67%提升至88.88%，并在所有基线模型上实现了十倍的利润增长，即使对未经过对齐的模型也是如此。