As recommender systems become widely deployed in different domains, they increasingly influence their users' beliefs and preferences. Auditing recommender systems is crucial as it not only ensures the continuous improvement of recommendation algorithms but also safeguards against potential issues like biases and ethical concerns. In this paper, we view recommender system auditing from a causal lens and provide a general recipe for defining auditing metrics. Under this general causal auditing framework, we categorize existing auditing metrics and identify gaps in them -- notably, the lack of metrics for auditing user agency while accounting for the multi-step dynamics of the recommendation process. We leverage our framework and propose two classes of such metrics:future- and past-reacheability and stability, that measure the ability of a user to influence their own and other users' recommendations, respectively. We provide both a gradient-based and a black-box approach for computing these metrics, allowing the auditor to compute them under different levels of access to the recommender system. In our experiments, we demonstrate the efficacy of methods for computing the proposed metrics and inspect the design of recommender systems through these proposed metrics.

翻译：随着推荐系统在不同领域的广泛应用，它们日益影响着用户的信念与偏好。对推荐系统进行审计至关重要，这不仅有助于持续改进推荐算法，还能防范偏见与伦理问题等潜在风险。本文从因果视角审视推荐系统审计，并提出定义审计指标的通用方法。在此统一的因果审计框架下，我们对现有审计指标进行分类，并指出其存在的不足——特别是缺乏在考虑推荐过程多步动态性的同时评估用户自主性的指标。基于该框架，我们提出两类新指标：未来/过去可达性与稳定性，分别用于衡量用户影响自身及其他用户推荐结果的能力。我们提供了基于梯度的计算方法和黑盒计算方法，使审计者能够根据对推荐系统的不同访问权限计算这些指标。实验部分验证了所提指标计算方法的有效性，并通过这些指标对推荐系统的设计进行了深入分析。