Gender-fair language, an evolving German linguistic variation, fosters inclusion by addressing all genders or using neutral forms. Nevertheless, there is a significant lack of resources to assess the impact of this linguistic shift on classification using language models (LMs), which are probably not trained on such variations. To address this gap, we present Lou, the first dataset featuring high-quality reformulations for German text classification covering seven tasks, like stance detection and toxicity classification. Evaluating 16 mono- and multi-lingual LMs on Lou shows that gender-fair language substantially impacts predictions by flipping labels, reducing certainty, and altering attention patterns. However, existing evaluations remain valid, as LM rankings of original and reformulated instances do not significantly differ. While we offer initial insights on the effect on German text classification, the findings likely apply to other languages, as consistent patterns were observed in multi-lingual and English LMs.

翻译：性别公平语言作为一种不断发展的德语语言变体，通过涵盖所有性别或使用中性形式来促进包容性。然而，目前严重缺乏评估这种语言变化对使用语言模型（LMs）进行分类的影响的资源，这些模型很可能未在此类变体上进行训练。为填补这一空白，我们提出了Lou——首个涵盖立场检测和毒性分类等七项任务的德语文本分类高质量重述数据集。通过对16个单语和多语LMs在Lou上的评估表明，性别公平语言会通过翻转标签、降低确定性和改变注意力模式等方式显著影响预测结果。然而，现有评估方法仍然有效，因为LMs对原始实例和重述实例的排序并未出现显著差异。虽然我们针对德语文本分类的影响提供了初步见解，但由于在多语和英语LMs中观察到一致的模式，这些发现很可能适用于其他语言。