Real-world domain experts (e.g., doctors) rarely annotate only a decision label in their day-to-day workflow without providing explanations. Yet, existing low-resource learning techniques, such as Active Learning (AL), that aim to support human annotators mostly focus on the label while neglecting the natural language explanation of a data point. This work proposes a novel AL architecture to support experts' real-world need for label and explanation annotations in low-resource scenarios. Our AL architecture leverages an explanation-generation model to produce explanations guided by human explanations, a prediction model that utilizes generated explanations toward prediction faithfully, and a novel data diversity-based AL sampling strategy that benefits from the explanation annotations. Automated and human evaluations demonstrate the effectiveness of incorporating explanations into AL sampling and the improved human annotation efficiency and trustworthiness with our AL architecture. Additional ablation studies illustrate the potential of our AL architecture for transfer learning, generalizability, and integration with large language models (LLMs). While LLMs exhibit exceptional explanation-generation capabilities for relatively simple tasks, their effectiveness in complex real-world tasks warrants further in-depth study.

翻译：真实世界中的领域专家（如医生）在日常工作中极少仅标注决策标签而不提供解释。然而，现有的低资源学习技术（如主动学习）虽旨在支持人工标注者，却主要关注标签而忽略了数据点的自然语言解释。本研究提出一种新颖的主动学习架构，以满足专家在低资源场景下对标签和解释标注的实际需求。该主动学习架构利用解释生成模型生成由人类解释引导的解释，一个预测模型忠实利用生成解释进行预测，以及一种基于数据多样性的新型主动学习采样策略，该策略受益于解释标注。自动评估和人工评估表明，将解释融入主动学习采样具有有效性，且该主动学习架构能提升人工标注效率与可信度。额外的消融研究展示了本架构在迁移学习、泛化能力以及与大型语言模型集成方面的潜力。尽管大型语言模型在相对简单任务中展现出卓越的解释生成能力，但其在复杂真实世界任务中的有效性仍需进一步深入研究。