Though ML practitioners increasingly employ various Responsible ML (RML) strategies, their methodological approach in practice is still unclear. In particular, the constraints, assumptions, and choices of practitioners with technical duties -- such as developers, engineers, and data scientists -- are often implicit, subtle, and under-scrutinized in HCI and related fields. We interviewed 22 technically oriented ML practitioners across seven domains to understand the characteristics of their methodological approaches to RML through the lens of ideal and non-ideal theorizing of fairness. We find that practitioners' methodological approaches fall along a spectrum of idealization. While they structured their approaches through ideal theorizing, such as by abstracting RML workflow from the inquiry of applicability of ML, they did not pay deliberate attention and systematically documented their non-ideal approaches, such as diagnosing imperfect conditions. We end our paper with a discussion of a new methodological approach, inspired by elements of non-ideal theory, to structure technical practitioners' RML process and facilitate collaboration with other stakeholders.
翻译:尽管机器学习从业者越来越多地采用各种负责任的机器学习(RML)策略,但他们在实践中的方法论仍不明确。特别是,在人机交互及相关领域,技术从业者(如开发者、工程师和数据科学家)所面临的约束、假设及选择往往隐晦、微妙且缺乏充分审视。我们通过理想与非理想公平理论视角,对七个领域的22名技术导向型机器学习从业者进行了访谈,以了解其RML方法论特征。研究发现,从业者的方法论处于理想化光谱之中:他们通过理想理论构建方法框架(例如将RML工作流与机器学习适用性探究相分离),但未能有意识地关注或系统记录非理想方法(如对不完美条件的诊断)。本文最后提出一种受非理想理论元素启发的新方法论框架,用以结构化技术从业者的RML流程,并促进与其他利益相关方的协作。