Machine learning (ML) enabled classification models are becoming increasingly popular for tackling the sheer volume and speed of online misinformation and other content that could be identified as harmful. In building these models, data scientists need to take a stance on the legitimacy, authoritativeness and objectivity of the sources of ``truth" used for model training and testing. This has political, ethical and epistemic implications which are rarely addressed in technical papers. Despite (and due to) their reported high accuracy and performance, ML-driven moderation systems have the potential to shape online public debate and create downstream negative impacts such as undue censorship and the reinforcing of false beliefs. Using collaborative ethnography and theoretical insights from social studies of science and expertise, we offer a critical analysis of the process of building ML models for (mis)information classification: we identify a series of algorithmic contingencies--key moments during model development that could lead to different future outcomes, uncertainty and harmful effects as these tools are deployed by social media platforms. We conclude by offering a tentative path toward reflexive and responsible development of ML tools for moderating misinformation and other harmful content online.

翻译：基于机器学习的分类模型正日益流行，用于应对网络错误信息及其他可能被判定为有害的内容的数量庞大、传播迅速的问题。在构建这些模型时，数据科学家需要对用于模型训练和测试的"真相"来源的合法性、权威性和客观性表明立场。这涉及政治、伦理和认识论层面的影响，而这些在技术论文中鲜有提及。尽管（且由于）其报告的高准确率与性能，机器学习驱动的审核系统有可能塑造网络公共讨论，并引发下游负面影响，例如不当审查以及强化错误信念。通过协作民族志方法以及融合科学专业知识社会研究的理论洞见，我们对构建用于（错误）信息分类的机器学习模型的过程进行了批判性分析：我们识别出一系列算法偶然性——即模型开发过程中可能导致不同未来结果、不确定性及有害影响（当这些工具被社交媒体平台部署时）的关键时刻。最后，我们提出了一条迈向反思性与负责任开发用于审核网络错误信息及其他有害内容的机器学习工具的初步路径。