Bargaining is an important and unique part of negotiation between humans. As LLM-driven agents learn to negotiate and act like real humans, how to evaluate agents' bargaining abilities remains an open problem. For the first time, we formally described the Bargaining task as an asymmetric incomplete information game, defining the gains of the Buyer and Seller in multiple bargaining processes. It allows us to quantitatively assess an agent's performance in the Bargain task. We collected a real product price dataset, AmazonHistoryPrice, and conducted evaluations of various LLM agents' bargaining abilities. We find that playing a Buyer is much harder than a Seller, and increasing model size can not effectively improve the Buyer's performance. To address the challenge, we propose a novel approach called OG-Narrator that integrates a deterministic Offer Generator to control the price range of Buyer's offers, and an LLM Narrator to create natural language sentences for generated offers. Experimental results show that OG-Narrator improves the buyer's deal rates from 26.67% to 88.88% and brings a ten times of multiplication of profits on all baselines, even a model that has not been aligned.

翻译：议价是人类谈判中重要且独特的部分。随着大语言模型驱动的智能体学会像真实人类一样谈判和行动，如何评估智能体的议价能力仍是一个开放性问题。我们首次将议价任务形式化为非对称不完全信息博弈，定义了多次议价过程中买方和卖方的收益，从而能够定量评估智能体在议价任务中的表现。我们收集了真实产品价格数据集AmazonHistoryPrice，并对多种大语言模型智能体的议价能力进行了评估。研究发现，扮演买方比卖方困难得多，且增大模型规模无法有效提升买方的表现。为应对这一挑战，我们提出了一种名为OG-Narrator的新方法，该方法集成了确定性出价生成器以控制买方出价的价格范围，并利用大语言模型叙述器为生成的出价创建自然语言句子。实验结果表明，OG-Narrator将买方的成交率从26.67%提升至88.88%，并在所有基线上实现了十倍的利润增长，甚至对未经对齐的模型也有效。