Conversational Recommendation Systems (CRSs) have recently started to leverage pretrained language models (LM) such as BERT for their ability to semantically interpret a wide range of preference statement variations. However, pretrained LMs are well-known to be prone to intrinsic biases in their training data, which may be exacerbated by biases embedded in domain-specific language data(e.g., user reviews) used to fine-tune LMs for CRSs. We study a recently introduced LM-driven recommendation backbone (termed LMRec) of a CRS to investigate how unintended bias i.e., language variations such as name references or indirect indicators of sexual orientation or location that should not affect recommendations manifests in significantly shifted price and category distributions of restaurant recommendations. The alarming results we observe strongly indicate that LMRec has learned to reinforce harmful stereotypes through its recommendations. For example, offhand mention of names associated with the black community significantly lowers the price distribution of recommended restaurants, while offhand mentions of common male-associated names lead to an increase in recommended alcohol-serving establishments. These and many related results presented in this work raise a red flag that advances in the language handling capability of LM-drivenCRSs do not come without significant challenges related to mitigating unintended bias in future deployed CRS assistants with a potential reach of hundreds of millions of end-users.

翻译：最近已开始利用事先培训的语言模型,如BERT, 以利用事先培训的语言模型(LM),如BERT, 以便其能够对广泛的优惠声明差异进行语义解释;然而,众所周知,事先培训的语言模型在其培训数据中容易出现内在偏见,而用于微调CRS使用的语言数据(例如用户审查)中所含的偏差可能加剧这种偏见;我们研究最近推出的CRS中由LM(LMREc)驱动的一条建议主干线,以调查语言差异,如名称引用或性取向或地点的间接指标等不应影响建议的语言变化,其表现在价格和餐馆建议分配类别上出现显著变化;我们观察到的惊人结果强烈表明,LMERc通过建议学会学会强化有害的陈规定型观念;例如,间接提及与黑人社区有关的名字大大降低了推荐的餐馆的价格分布;同时不直接提及普通的男性关联名称导致推荐的酒精维护机构增加。