When faced with a polar question, speakers often provide overinformative answers going beyond a simple "yes" or "no". But what principles guide the selection of additional information? In this paper, we provide experimental evidence from two studies suggesting that overinformativeness in human answering is driven by considerations of relevance to the questioner's goals which they flexibly adjust given the functional context in which the question is uttered. We take these human results as a strong benchmark for investigating question-answering performance in state-of-the-art neural language models, conducting an extensive evaluation on items from human experiments. We find that most models fail to adjust their answering behavior in a human-like way and tend to include irrelevant information. We show that GPT-3 is highly sensitive to the form of the prompt and only achieves human-like answer patterns when guided by an example and cognitively-motivated explanation.

翻译：面对极性问句时，说话者常提供超出简单"是"或"否"的过度信息型回答。但何种原则指导着额外信息的选择？本文通过两项研究提供实验证据，表明人类回答中的过度信息现象受制于对提问者目标相关性的考量，且这种考量会根据问题所处的功能性语境灵活调整。我们将这些人类实验结果作为强基准，系统评估最先进神经语言模型在人类实验项目中的问答表现，发现多数模型未能以类人方式调整回答行为，且倾向于包含无关信息。研究表明，GPT-3对提示形式高度敏感，仅当配以示例和认知驱动的解释性引导时，才能呈现类人回答模式。