Machine learning technology has become ubiquitous, but, unfortunately, often exhibits bias. As a consequence, disparate stakeholders need to interact with and make informed decisions about using machine learning models in everyday systems. Visualization technology can support stakeholders in understanding and evaluating trade-offs between, for example, accuracy and fairness of models. This paper aims to empirically answer "Can visualization design choices affect a stakeholder's perception of model bias, trust in a model, and willingness to adopt a model?" Through a series of controlled, crowd-sourced experiments with more than 1,500 participants, we identify a set of strategies people follow in deciding which models to trust. Our results show that men and women prioritize fairness and performance differently and that visual design choices significantly affect that prioritization. For example, women trust fairer models more often than men do, participants value fairness more when it is explained using text than as a bar chart, and being explicitly told a model is biased has a bigger impact than showing past biased performance. We test the generalizability of our results by comparing the effect of multiple textual and visual design choices and offer potential explanations of the cognitive mechanisms behind the difference in fairness perception and trust. Our research guides design considerations to support future work developing visualization systems for machine learning.

翻译：机器学习技术已变得无处不在，但不幸的是，常常表现出偏见。因此，不同利益相关者需要与日常系统中的机器学习模型进行交互，并做出明智的使用决策。可视化技术可以支持利益相关者理解和评估模型在准确性与公平性等方面的权衡。本文旨在通过实证研究回答：“可视化设计选择是否会影响利益相关者对模型偏见的感知、对模型的信任以及采用模型的意愿？”通过一系列受控的众包实验，涉及超过1500名参与者，我们识别出人们在决定信任哪些模型时所遵循的一组策略。结果表明，男性和女性对公平性和性能的优先级排序存在差异，且视觉设计选择显著影响这种优先级排序。例如，女性比男性更倾向于信任更公平的模型；当公平性用文字解释时，参与者比用条形图展示时更重视公平性；直接告知模型存在偏见比展示过去存在偏见的表现影响更大。我们通过比较多种文本和视觉设计选择的效果来检验结果的普适性，并提供了对公平性感知和信任差异背后认知机制的可能解释。我们的研究为支持未来开发机器学习可视化系统的设计考量提供了指导。