Unfair behaviors of Machine Learning (ML) software have garnered increasing attention and concern among software engineers. To tackle this issue, extensive research has been dedicated to conducting fairness testing of ML software, and this paper offers a comprehensive survey of existing studies in this field. We collect 100 papers and organize them based on the testing workflow (i.e., how to test) and testing components (i.e., what to test). Furthermore, we analyze the research focus, trends, and promising directions in the realm of fairness testing. We also identify widely-adopted datasets and open-source tools for fairness testing.

翻译：机器学习（ML）软件的不公平行为日益引起软件工程师的关注与担忧。为应对这一问题，学界已投入大量研究开展ML软件的公平性测试，本文对该领域的现有研究进行了全面综述。我们收集了100篇论文，并依据测试工作流（即如何测试）和测试组件（即测试什么）对其进行分类整理。此外，我们分析了公平性测试领域的研究重点、发展趋势及潜在方向，并识别了该领域广泛采用的数据集与开源工具。