As artificial intelligence (AI) increasingly becomes an integral part of our societal and individual activities, there is a growing imperative to develop responsible AI solutions. Despite a diverse assortment of machine learning fairness solutions is proposed in the literature, there is reportedly a lack of practical implementation of these tools in real-world applications. Industry experts have participated in thorough discussions on the challenges associated with operationalising fairness in the development of machine learning-empowered solutions, in which a shift toward human-centred approaches is promptly advocated to mitigate the limitations of existing techniques. In this work, we propose a human-in-the-loop approach for fairness auditing, presenting a mixed visual analytical system (hereafter referred to as 'FairCompass'), which integrates both subgroup discovery technique and the decision tree-based schema for end users. Moreover, we innovatively integrate an Exploration, Guidance and Informed Analysis loop, to facilitate the use of the Knowledge Generation Model for Visual Analytics in FairCompass. We evaluate the effectiveness of FairCompass for fairness auditing in a real-world scenario, and the findings demonstrate the system's potential for real-world deployability. We anticipate this work will address the current gaps in research for fairness and facilitate the operationalisation of fairness in machine learning systems.

翻译：随着人工智能日益成为我们社会和个体活动不可或缺的一部分，开发负责任的人工智能解决方案变得愈发迫切。尽管文献中提出了多种多样的机器学习公平性解决方案，但据报道，这些工具在实际应用中的实践部署仍然不足。行业专家深入讨论了在开发机器学习赋能解决方案时操作化公平性所面临的挑战，其中及时倡导转向以人为中心的方法，以缓解现有技术的局限性。在这项工作中，我们提出了一种人在回路的公平性审计方法，展示了一个混合视觉分析系统（以下简称“公平指南针”），该集成了子组发现技术和基于决策树的方案，供最终用户使用。此外，我们创新性地整合了探索、引导与知情分析循环，以促进知识生成模型在公平指南针中用于视觉分析。我们在一个真实场景中评估了公平指南针进行公平性审计的有效性，结果表明该系统具有实际部署的潜力。我们预期这项工作将填补当前公平性研究中的空白，并促进机器学习系统中公平性的操作化。