Neural language models, which reach state-of-the-art results on most natural language processing tasks, are trained on large text corpora that inevitably contain value-burdened content and often capture undesirable biases, which the models reflect. This case study focuses on the political biases of pre-trained encoders in Czech and compares them with a representative value survey. Because Czech is a gendered language, we also measure how the grammatical gender coincides with responses to men and women in the survey. We introduce a novel method for measuring the model's perceived political values. We find that the models do not assign statement probability following value-driven reasoning, and there is no systematic difference between feminine and masculine sentences. We conclude that BERT-sized models do not manifest systematic alignment with political values and that the biases observed in the models are rather due to superficial imitation of training data patterns than systematic value beliefs encoded in the models.

翻译：神经语言模型在大多数自然语言处理任务中达到了最先进的结果，这些模型是在包含价值负载内容的大型文本语料库上训练的，往往会捕捉到不良偏见，而这些模型也会反映这些偏见。本案例研究聚焦于捷克预训练编码器的政治偏见，并将其与一项代表性价值观调查进行比较。由于捷克语是一种有性别区分的语言，我们还测量了语法性别如何与调查中男性和女性的回答相吻合。我们引入了一种新颖的方法来测量模型感知到的政治价值观。我们发现，模型在分配语句概率时并非遵循价值驱动的推理，并且女性化和男性化句子之间没有系统性差异。我们得出结论，BERT规模大小的模型并未表现出与政治价值观的系统性对齐，而模型中观察到的偏见更可能是由于对训练数据模式的表面模仿，而非模型内编码的系统性价值信念。