Headline generation, a key task in abstractive summarization, strives to condense a full-length article into a succinct, single line of text. Notably, while contemporary encoder-decoder models excel based on the ROUGE metric, they often falter when it comes to the precise generation of numerals in headlines. We identify the lack of datasets providing fine-grained annotations for accurate numeral generation as a major roadblock. To address this, we introduce a new dataset, the NumHG, and provide over 27,000 annotated numeral-rich news articles for detailed investigation. Further, we evaluate five well-performing models from previous headline generation tasks using human evaluation in terms of numerical accuracy, reasonableness, and readability. Our study reveals a need for improvement in numerical accuracy, demonstrating the potential of the NumHG dataset to drive progress in number-focused headline generation and stimulate further discussions in numeral-focused text generation.

翻译：标题生成作为抽象式摘要中的关键任务，旨在将完整文章压缩为简洁的单行文本。值得注意的是，尽管当代编码器-解码器模型在ROUGE指标上表现优异，但在标题中数字的精确生成方面常常表现不佳。我们指出，缺乏提供细粒度标注以支持准确数字生成的数据集是主要障碍。为解决这一问题，我们提出了新数据集NumHG，并提供了超过27,000篇标注丰富的数字密集型新闻文章以进行详细研究。此外，我们从数字准确性、合理性和可读性角度，通过人工评估对先前标题生成任务中的五个高性能模型进行了评测。研究表明数字准确性亟需改进，证明了NumHG数据集在推动数字聚焦标题生成进展方面的潜力，并有望激发数字聚焦文本生成领域的进一步讨论。