People are relying on AI agents to assist them with various tasks. The human must know when to rely on the agent, collaborate with the agent, or ignore its suggestions. In this work, we propose to learn rules grounded in data regions and described in natural language that illustrate how the human should collaborate with the AI. Our novel region discovery algorithm finds local regions in the data as neighborhoods in an embedding space that corrects the human prior. Each region is then described using an iterative and contrastive procedure where a large language model describes the region. We then teach these rules to the human via an onboarding stage. Through user studies on object detection and question-answering tasks, we show that our method can lead to more accurate human-AI teams. We also evaluate our region discovery and description algorithms separately.

翻译：人们正依赖人工智能代理协助完成各类任务。人类必须知道何时依赖代理、何时与其协作、何时忽略其建议。本文提出一种学习规则的方法，这些规则基于数据区域并以自然语言描述，旨在阐明人类应如何与AI协作。我们创新的区域发现算法能够在数据中定位局部区域，这些区域作为嵌入空间中的邻域，能修正人类先验知识。随后通过迭代对比流程，利用大语言模型对每个区域进行描述。最后通过引导阶段将这些规则传授给人类。基于目标检测与问答任务的用户研究表明，我们的方法能构建更准确的人机协作团队。我们还分别评估了区域发现与描述算法的性能。