The field of Natural Language Generation (NLG) suffers from a severe shortage of labeled data due to the extremely expensive and time-consuming process involved in manual annotation. A natural approach for coping with this problem is active learning (AL), a well-known machine learning technique for improving annotation efficiency by selectively choosing the most informative examples to label. However, while AL has been well-researched in the context of text classification, its application to NLG remains largely unexplored. In this paper, we present a first systematic study of active learning for NLG, considering a diverse set of tasks and multiple leading selection strategies, and harnessing a strong instruction-tuned model. Our results indicate that the performance of existing AL strategies is inconsistent, surpassing the baseline of random example selection in some cases but not in others. We highlight some notable differences between the classification and generation scenarios, and analyze the selection behaviors of existing AL strategies. Our findings motivate exploring novel approaches for applying AL to generation tasks.

翻译：自然语言生成（NLG）领域因人工标注过程极其昂贵且耗时，面临严重的标注数据短缺问题。应对这一问题的自然方法是采用主动学习（AL），这是一种通过选择性挑选最具信息量的样本进行标注、从而提高标注效率的经典机器学习技术。然而，尽管主动学习在文本分类任务中已得到深入研究，但其在自然语言生成中的应用仍鲜有探索。本文首次系统研究了面向自然语言生成的主动学习问题，考虑了多样化的任务类型、多种主流选择策略，并采用了经过指令调优的强模型。研究结果表明，现有主动学习策略的表现具有不一致性：在某些情况下能超越随机样本选择的基线，但在其他情况下则不然。我们重点阐释了分类场景与生成场景之间的显著差异，并分析了现有主动学习策略的选择行为。这些发现为探索将主动学习应用于生成任务的新方法提供了研究动机。