Transformer-based models have consistently produced substantial performance gains across a variety of NLP tasks, compared to shallow models. However, deep models are orders of magnitude more computationally expensive than shallow models, especially on tasks with large sequence lengths, such as document-level event detection. In this work, we attempt to bridge the performance gap between shallow and deep models on document-level event detection by using abstractive text summarization as an augmentation method. We augment the DocEE dataset by generating abstractive summaries of examples from low-resource classes. For classification, we use linear SVM with TF-IDF representations and RoBERTa-base. We use BART for zero-shot abstractive summarization, making our augmentation setup less resource-intensive compared to supervised fine-tuning. We experiment with four decoding methods for text generation, namely beam search, top-k sampling, top-p sampling, and contrastive search. Furthermore, we investigate the impact of using document titles as additional input for classification. Our results show that using the document title offers 2.04% and 3.19% absolute improvement in macro F1-score for linear SVM and RoBERTa, respectively. Augmentation via summarization further improves the performance of linear SVM by about 0.5%, varying slightly across decoding methods. Overall, our augmentation setup yields insufficient improvements for linear SVM compared to RoBERTa.

翻译：Transformer模型在各种自然语言处理任务中持续展现出优于浅层模型的显著性能提升。然而，深度模型在计算复杂度上比浅层模型高出数个数量级，尤其在处理长序列任务（如文档级事件检测）时更为突出。本研究尝试通过将抽象文本摘要作为增强方法，弥合浅层与深度模型在文档级事件检测上的性能差距。我们通过为低资源类别的样本生成抽象摘要，扩充了DocEE数据集。分类时，我们采用基于TF-IDF表示的线性支持向量机与RoBERTa-base模型，并使用BART进行零样本抽象摘要生成，从而使增强方案相比监督微调减少了资源消耗。我们在文本生成中实验了四种解码方法：束搜索、Top-k采样、Top-p采样和对比搜索。此外，我们探究了将文档标题作为额外输入对分类的影响。结果表明，使用文档标题使线性支持向量机和RoBERTa的宏F1分数分别绝对提升了2.04%和3.19%。通过摘要增强进一步使线性支持向量机性能提升约0.5%，不同解码方法间略有差异。总体而言，与RoBERTa相比，我们的增强方案对线性支持向量机的改进效果有限。