Emotion corpora are typically sampled based on keyword/hashtag search or by asking study participants to generate textual instances. In any case, these corpora are not uniform samples representing the entirety of a domain. We hypothesize that this practice of data acquisition leads to unrealistic correlations between overrepresented topics in these corpora that harm the generalizability of models. Such topic bias could lead to wrong predictions for instances like "I organized the service for my aunt's funeral." when funeral events are over-represented for instances labeled with sadness, despite the emotion of pride being more appropriate here. In this paper, we study this topic bias both from the data and the modeling perspective. We first label a set of emotion corpora automatically via topic modeling and show that emotions in fact correlate with specific topics. Further, we see that emotion classifiers are confounded by such topics. Finally, we show that the established debiasing method of adversarial correction via gradient reversal mitigates the issue. Our work points out issues with existing emotion corpora and that more representative resources are required for fair evaluation of models predicting affective concepts from text.

翻译：情感语料库通常基于关键词/话题标签搜索生成，或要求研究参与者提供文本实例。无论如何，这些语料库并非代表整个领域的均匀样本。我们假设这种数据采集方法会导致语料库中过度呈现的主题间存在非现实关联，从而损害模型的泛化能力。这种主题偏差可能导致对诸如“我为我姑姑的葬礼组织了仪式”这样的实例产生错误预测——虽然“骄傲”情感在此更恰当，但葬礼事件在标记为“悲伤”的实例中过度呈现。本文从数据和建模两个角度研究这种主题偏差。首先通过主题建模自动标注一组情感语料库，证明情感实际上与特定主题相关。此外，我们发现情感分类器会受到此类主题的混淆。最后，我们证明通过梯度反转进行对抗性纠正这一既定去偏方法能够缓解该问题。本研究指出了现有情感语料库存在的问题，并提出需要更具代表性的资源来公平评估从文本预测情感概念的模型。