We introduce AmbigNLG, a novel task designed to tackle the challenge of task ambiguity in instructions for Natural Language Generation (NLG). Ambiguous instructions often impede the performance of Large Language Models (LLMs), especially in complex NLG tasks. To tackle this issue, we propose an ambiguity taxonomy that categorizes different types of instruction ambiguities and refines initial instructions with clearer specifications. Accompanying this task, we present AmbigSNI-NLG, a dataset comprising 2,500 instances annotated to facilitate research in AmbigNLG. Through comprehensive experiments with state-of-the-art LLMs, we demonstrate that our method significantly enhances the alignment of generated text with user expectations, achieving up to a 15.02-point increase in ROUGE scores. Our findings highlight the critical importance of addressing task ambiguity to fully harness the capabilities of LLMs in NLG tasks. Furthermore, we confirm the effectiveness of our method in practical settings involving interactive ambiguity mitigation with users, underscoring the benefits of leveraging LLMs for interactive clarification.

翻译：本文提出AmbigNLG这一新颖任务，旨在应对自然语言生成（NLG）指令中任务模糊性带来的挑战。模糊指令通常会阻碍大型语言模型（LLMs）的性能表现，尤其在复杂的NLG任务中更为明显。为解决该问题，我们提出一种模糊性分类体系，用于对不同类型的指令模糊性进行分类，并通过更明确的规范对初始指令进行优化。配合此任务，我们构建了AmbigSNI-NLG数据集，其中包含2,500条标注实例，以促进AmbigNLG相关研究。通过对前沿LLMs进行系统实验，我们证明该方法能显著提升生成文本与用户期望的匹配度，在ROUGE指标上最高可获得15.02分的提升。我们的研究结果凸显了解决任务模糊性对于充分发挥LLMs在NLG任务中潜力的关键作用。此外，我们验证了该方法在涉及用户交互式模糊消解的实际场景中的有效性，证实了利用LLMs进行交互式澄清的显著优势。