Due to their similarity-based learning objectives, pretrained sentence encoders often internalize stereotypical assumptions that reflect the social biases that exist within their training corpora. In this paper, we describe several kinds of stereotypes concerning different communities that are present in popular sentence representation models, including pretrained next sentence prediction and contrastive sentence representation models. We compare such models to textual entailment models that learn language logic for a variety of downstream language understanding tasks. By comparing strong pretrained models based on text similarity with textual entailment learning, we conclude that the explicit logic learning with textual entailment can significantly reduce bias and improve the recognition of social communities, without an explicit de-biasing process

翻译：由于其基于相似性的学习目标,未经训练的句子编解者往往将反映其培训公司内部存在的社会偏见的陈规定型假设内化,在本文件中,我们描述了流行句代表模式中存在的关于不同社区的几种陈规定型,包括未经训练的下一句预测和对比式判刑代表模式。我们把这些模式与学习各种下游语言理解任务的语言逻辑的文字要求模型相比较。我们比较了基于文本与文字要求学习相似的经过训练的强有力模型,我们的结论是,与文字要求有关的明确逻辑学习可以大大减少偏见,改善社会社区的承认,而没有明确的消除偏见进程。</s>