Building on recent advances in AI, hybrid decision making (HDM) holds the promise of improving human decision quality and reducing cognitive load. We work in the context of learning to guide (LtG), a recently proposed HDM framework in which the human is always responsible for the final decision: rather than suggesting decisions, in LtG the AI supplies (textual) guidance useful for facilitating decision making. One limiting factor of existing approaches is that their guidance compounds information about all possible outcomes, and as a result it can be difficult to digest. We address this issue by introducing ConfGuide, a novel LtG approach that generates more succinct and targeted guidance. To this end, it employs conformal risk control to select a set of outcomes, ensuring a cap on the false negative rate. We demonstrate our approach on a real-world multi-label medical diagnosis task. Our empirical evaluation highlights the promise of ConfGuide.
翻译:在人工智能最新进展的基础上,混合决策有望提升人类决策质量并减轻认知负荷。我们聚焦于近期提出的"学习引导"框架——在该框架中,人类始终对最终决策负责:人工智能不提供决策建议,而是提供有助于决策的(文本)指导。现有方法的一个局限在于,其引导信息混杂了所有可能结果的信息,导致用户难以消化。为解决该问题,我们提出ConfGuide——一种能生成更简洁且针对性更强的引导的新型学习引导方法。该方法通过应用共形风险控制选择一组结果,确保误报率上限可控。我们在真实世界的多标签医疗诊断任务中验证了该方法,实验评估凸显了ConfGuide的应用潜力。