The wealth of text data generated by social media has enabled new kinds of analysis of emotions with language models. These models are often trained on small and costly datasets of text annotations produced by readers who guess the emotions expressed by others in social media posts. This affects the quality of emotion identification methods due to training data size limitations and noise in the production of labels used in model development. We present LEIA, a model for emotion identification in text that has been trained on a dataset of more than 6 million posts with self-annotated emotion labels for happiness, affection, sadness, anger, and fear. LEIA is based on a word masking method that enhances the learning of emotion words during model pre-training. LEIA achieves macro-F1 values of approximately 73 on three in-domain test datasets, outperforming other supervised and unsupervised methods in a strong benchmark that shows that LEIA generalizes across posts, users, and time periods. We further perform an out-of-domain evaluation on five different datasets of social media and other sources, showing LEIA's robust performance across media, data collection methods, and annotation schemes. Our results show that LEIA generalizes its classification of anger, happiness, and sadness beyond the domain it was trained on. LEIA can be applied in future research to provide better identification of emotions in text from the perspective of the writer. The models produced for this article are publicly available at https://huggingface.co/LEIA

翻译：摘要：社交媒体生成的大量文本数据使得利用语言模型对情感进行新型分析成为可能。这类模型通常基于读者猜测社交媒体帖子中他人表达的情感而生成的文本标注数据集进行训练，但此类数据集规模小且成本高昂。受限于训练数据规模以及模型开发过程中标签生成的噪声，这影响了情感识别方法的质量。我们提出LEIA模型，用于文本情感识别，该模型基于一个包含超过600万条帖子且带有快乐、喜爱、悲伤、愤怒和恐惧五种自标注情感标签的数据集进行训练。LEIA采用一种词掩码方法，在模型预训练阶段增强情感词汇的学习。在三个领域内测试数据集上，LEIA的宏F1值达到约73，优于其他监督和无监督方法，并在强基准测试中表明LEIA能够跨帖子、用户和时间段进行泛化。我们进一步在五个来自社交媒体及其他来源的不同数据集上进行领域外评估，展示了LEIA在不同媒介、数据收集方法和标注方案下的稳健性能。结果表明，LEIA对愤怒、快乐和悲伤的分类泛化能力超越了其训练领域。LEIA可应用于未来研究，以从作者视角更好地识别文本中的情感。本文产生的模型已在https://huggingface.co/LEIA 公开发布。