Personal attacks in the context of social media conversations often lead to fast-paced derailment, leading to even more harmful exchanges being made. State-of-the-art systems for the detection of such conversational derailment often make use of deep learning approaches for prediction purposes. In this paper, we show that an Attention-based BERT architecture, pre-trained on a large Twitter corpus and fine-tuned on our task, is efficient and effective in making such predictions. This model shows clear advantages in performance to the existing LSTM model we use as a baseline. Additionally, we show that this impressive performance can be attained through fine-tuning on a relatively small, novel dataset, particularly after mitigating overfitting issues through synthetic oversampling techniques. By introducing the first transformer based model for forecasting conversational events on Twitter, this work lays the foundation for a practical tool to encourage better interactions on one of the most ubiquitous social media platforms.

翻译：在社交媒体对话中，人身攻击往往导致对话快速偏离正轨，进而引发更多有害言论的交换。现有用于检测此类对话偏离的先进系统通常采用深度学习方法进行预测。在本文中，我们展示了一种基于注意力机制的BERT架构——该模型在大型Twitter语料库上进行预训练，并在我们的任务上进行微调——能够高效且有效地进行此类预测。该模型相较于作为基线的现有LSTM模型，在性能上展现出明显优势。此外，我们证明，通过在大规模合成过采样技术缓解过拟合问题后，仅借助相对较小、新颖的数据集进行微调即可实现这一卓越性能。通过引入首个基于Transformer的Twitter对话事件预测模型，本研究为构建实用工具奠定了基础，从而鼓励在这个最普遍的社交媒体平台上实现更良性的互动。