Albeit the widespread application of recommender systems (RecSys) in our daily lives, rather limited research has been done on quantifying unfairness and biases present in such systems. Prior work largely focuses on determining whether a RecSys is discriminating or not but does not compute the amount of bias present in these systems. Biased recommendations may lead to decisions that can potentially have adverse effects on individuals, sensitive user groups, and society. Hence, it is important to quantify these biases for fair and safe commercial applications of these systems. This paper focuses on quantifying popularity bias that stems directly from the output of RecSys models, leading to over recommendation of popular items that are likely to be misaligned with user preferences. Four metrics to quantify popularity bias in RescSys over time in dynamic setting across different sensitive user groups have been proposed. These metrics have been demonstrated for four collaborative filtering based RecSys algorithms trained on two commonly used benchmark datasets in the literature. Results obtained show that the metrics proposed provide a comprehensive understanding of growing disparities in treatment between sensitive groups over time when used conjointly.

翻译：尽管推荐系统已广泛应用于日常生活，但对其不公正性与偏差的量化研究仍相当有限。既有工作主要集中于判定推荐系统是否存在歧视，却未计算这些系统中偏差的实际程度。有偏差的推荐可能导致对个体、敏感用户群体及社会产生潜在负面影响的决策。因此，为保障这些系统的公平与安全商业应用，量化此类偏差至关重要。本文聚焦于量化直接源于推荐模型输出的流行度偏差——这种偏差会导致过度推荐与用户偏好可能不符的热门物品。我们提出了四项度量指标，用于在动态环境下跨不同敏感用户群体量化推荐系统的时变流行度偏差。这些指标已在两个公开基准数据集上训练的四种基于协同过滤的推荐算法中进行了验证。结果表明，联合使用所提指标可全面理解敏感群体间随时间推移而扩大的待遇差异。