Language models will inevitably err in situations with which they are unfamiliar. However, by effectively communicating uncertainties, they can still guide humans toward making sound decisions in those contexts. We demonstrate this idea by developing HEAR, a system that can successfully guide humans in simulated residential environments despite generating potentially inaccurate instructions. Diverging from systems that provide users with only the instructions they generate, HEAR warns users of potential errors in its instructions and suggests corrections. This rich uncertainty information effectively prevents misguidance and reduces the search space for users. Evaluation with 80 users shows that HEAR achieves a 13% increase in success rate and a 29% reduction in final location error distance compared to only presenting instructions to users. Interestingly, we find that offering users possibilities to explore, HEAR motivates them to make more attempts at the task, ultimately leading to a higher success rate. To our best knowledge, this work is the first to show the practical benefits of uncertainty communication in a long-horizon sequential decision-making problem.

翻译：语言模型在面对不熟悉的情境时不可避免地会出现错误。然而，通过有效传达不确定性，它们仍能在这些情境中引导人类做出合理决策。我们通过开发HEAR系统来验证这一理念，该系统尽管可能生成不准确的指令，但仍能在模拟居住环境中成功引导人类。与仅向用户提供生成指令的系统不同，HEAR会警告用户其指令中可能存在的错误并建议修正。这种丰富的置信度信息能有效防止误导并缩小用户的搜索空间。对80名用户的评估表明，与仅向用户呈现指令相比，HEAR实现了13%的成功率提升和29%的最终位置误差距离缩减。有趣的是，我们发现通过为用户提供探索可能性，HEAR激励他们进行更多任务尝试，最终达成更高的成功率。据我们所知，这项研究首次在长时域序贯决策问题中展示了置信度传达的实际效益。