Predictive student models are increasingly used in learning environments due to their ability to enhance educational outcomes and support stakeholders in making informed decisions. However, predictive models can be biased and produce unfair outcomes, leading to potential discrimination against some students and possible harmful long-term implications. This has prompted research on fairness metrics meant to capture and quantify such biases. Nonetheless, so far, existing fairness metrics used in education are predictive performance-oriented, focusing on assessing biased outcomes across groups of students, without considering the behaviors of the models nor the severity of the biases in the outcomes. Therefore, we propose a novel metric, the Model Absolute Density Distance (MADD), to analyze models' discriminatory behaviors independently from their predictive performance. We also provide a complementary visualization-based analysis to enable fine-grained human assessment of how the models discriminate between groups of students. We evaluate our approach on the common task of predicting student success in online courses, using several common predictive classification models on an open educational dataset. We also compare our metric to the only predictive performance-oriented fairness metric developed in education, ABROCA. Results on this dataset show that: (1) fair predictive performance does not guarantee fair models' behaviors and thus fair outcomes, (2) there is no direct relationship between data bias and predictive performance bias nor discriminatory behaviors bias, and (3) trained on the same data, models exhibit different discriminatory behaviors, according to different sensitive features too. We thus recommend using the MADD on models that show satisfying predictive performance, to gain a finer-grained understanding on how they behave and to refine models selection and their usage.

翻译：预测性学生模型因其能够增强教育成果并支持利益相关者做出明智决策，在学习环境中得到越来越广泛的应用。然而，预测模型可能存在偏差并产生不公平的结果，导致对某些学生的潜在歧视及可能的有害长期影响。这促使了旨在捕获并量化此类偏差的公平性指标的研究。尽管如此，迄今为止，教育领域使用的现有公平性指标均以预测性能为导向，专注于评估不同学生群体间的结果偏差，而未考虑模型的行为或结果中偏差的严重程度。因此，我们提出一种新指标——模型绝对密度距离（MADD），用于独立于预测性能分析模型的歧视性行为。我们还提供了一种基于可视化的补充分析，以实现对模型如何在学生群体间进行区分的细粒度人工评估。我们以预测在线课程中学生成绩的常见任务为例，使用几种常见的预测分类模型在开放教育数据集上评估了我们的方法。我们还将我们的指标与教育领域开发的唯一以预测性能为导向的公平性指标ABROCA进行了比较。在该数据集上的结果表明：（1）公平的预测性能并不能保证公平的模型行为，从而也不能保证公平的结果；（2）数据偏差与预测性能偏差或歧视性行为偏差之间没有直接关系；（3）在相同数据上训练的模型，根据不同的敏感特征，会表现出不同的歧视性行为。因此，我们建议对表现出令人满意预测性能的模型使用MADD，以获得对其行为更细粒度的理解，并优化模型选择及其使用。