Modern language models, while sophisticated, exhibit some inherent shortcomings, particularly in conversational settings. We claim that many of the observed shortcomings can be attributed to violation of one or more conversational principles. By drawing upon extensive research from both the social science and AI communities, we propose a set of maxims -- quantity, quality, relevance, manner, benevolence, and transparency -- for describing effective human-AI conversation. We first justify the applicability of the first four maxims (from Grice) in the context of human-AI interactions. We then argue that two new maxims, benevolence (concerning the generation of, and engagement with, harmful content) and transparency (concerning recognition of one's knowledge boundaries, operational constraints, and intents), are necessary for addressing behavior unique to modern human-AI interactions. The proposed maxims offer prescriptive guidance on how to assess conversational quality between humans and LLM-driven conversational agents, informing both their evaluation and improved design.
翻译:现代语言模型虽然复杂精妙,但在对话场景中仍表现出一些固有缺陷。我们认为,这些观测到的缺陷大多可归因于违反了一条或多条会话原则。通过借鉴社会科学和人工智能领域的广泛研究,我们提出了一套准则——数量、质量、关联性、方式、善意性和透明性——用于描述有效的人机对话。我们首先论证了前四条准则(源自格莱斯)在人机交互语境中的适用性,进而提出另外两条新准则——善良性(涉及有害内容的生成与互动)和透明性(涉及对自身知识边界、操作限制和意图的认知)——对于应对现代人机交互特有的行为是必要的。该准则体系为评估人类与大语言模型驱动的对话代理之间的对话质量提供了规范性指导,既适用于其评估环节,也为其优化设计提供参考。