Numerous fairness metrics have been proposed and employed by artificial intelligence (AI) experts to quantitatively measure bias and define fairness in AI models. Recognizing the need to accommodate stakeholders' diverse fairness understandings, efforts are underway to solicit their input. However, conveying AI fairness metrics to stakeholders without AI expertise, capturing their personal preferences, and seeking a collective consensus remain challenging and underexplored. To bridge this gap, we propose a new framework, EARN Fairness, which facilitates collective metric decisions among stakeholders without requiring AI expertise. The framework features an adaptable interactive system and a stakeholder-centered EARN Fairness process to Explain fairness metrics, Ask stakeholders' personal metric preferences, Review metrics collectively, and Negotiate a consensus on metric selection. To gather empirical results, we applied the framework to a credit rating scenario and conducted a user study involving 18 decision subjects without AI knowledge. We identify their personal metric preferences and their acceptable level of unfairness in individual sessions. Subsequently, we uncovered how they reached metric consensus in team sessions. Our work shows that the EARN Fairness framework enables stakeholders to express personal preferences and reach consensus, providing practical guidance for implementing human-centered AI fairness in high-risk contexts. Through this approach, we aim to harmonize fairness expectations of diverse stakeholders, fostering more equitable and inclusive AI fairness.

翻译：人工智能（AI）专家已提出并采用众多公平性指标来量化测量AI模型中的偏见并定义公平性。为适应利益相关者对公平性的多样化理解，当前正致力于征求他们的意见。然而，向不具备AI专业知识的利益相关者传达AI公平性指标、捕捉其个人偏好并寻求集体共识，仍然具有挑战性且研究不足。为弥合这一差距，我们提出一个新框架——EARN公平性，该框架促进利益相关者之间无需AI专业知识即可进行集体指标决策。该框架具备一个适应性交互系统和一个以利益相关者为中心的EARN公平性流程，用于解释公平性指标、询问利益相关者的个人指标偏好、集体评审指标，并就指标选择进行协商以达成共识。为收集实证结果，我们将该框架应用于信用评级场景，并开展了一项涉及18位无AI知识的决策主体的用户研究。我们在个体环节中识别了他们的个人指标偏好及其可接受的不公平程度。随后，我们揭示了他们在团队环节中如何达成指标共识。我们的研究表明，EARN公平性框架使利益相关者能够表达个人偏好并达成共识，为在高风险情境中实施以人为中心的AI公平性提供了实践指导。通过这一方法，我们旨在协调不同利益相关者的公平性期望，促进更公正、更具包容性的AI公平性。