Much attention and concern has been raised recently about bias and the use of machine learning algorithms in healthcare, especially as it relates to perpetuating racial discrimination and health disparities. Following an initial system dynamics workshop at the Data for Black Lives II conference hosted at MIT in January of 2019, a group of conference participants interested in building capabilities to use system dynamics to understand complex societal issues convened monthly to explore issues related to racial bias in AI and implications for health disparities through qualitative and simulation modeling. In this paper we present results and insights from the modeling process and highlight the importance of centering the discussion of data and healthcare on people and their experiences with healthcare and science, and recognizing the societal context where the algorithm is operating. Collective memory of community trauma, through deaths attributed to poor healthcare, and negative experiences with healthcare are endogenous drivers of seeking treatment and experiencing effective care, which impact the availability and quality of data for algorithms. These drivers have drastically disparate initial conditions for different racial groups and point to limited impact of focusing solely on improving diagnostic algorithms for achieving better health outcomes for some groups.

翻译：近年来，医疗保健领域中的偏见与机器学习算法应用引发广泛关注与担忧，尤其是其可能加剧种族歧视与健康差异的问题。2019年1月，在麻省理工学院举办的"黑人数据生命II"会议上，一场初步的系统动力学工作坊之后，一群有志于运用系统动力学理解复杂社会问题的会议参与者开始每月举行会议，通过定性与仿真建模方法探索人工智能中的种族偏见及其对健康差异的影响。本文呈现了建模过程中的成果与洞见，强调将数据与医疗保健讨论聚焦于人群及其医疗保健与科学经历的重要性，并识别算法运行所处的社会背景。社区创伤的集体记忆（源于医疗不良导致的死亡）以及医疗保健中的负面经历，构成寻求治疗与获得有效护理的内生驱动力，这些因素直接影响着算法可用数据的质量与数量。不同种族群体在这些驱动因素上的初始条件存在显著差异，表明仅专注于改进诊断算法以提升某些群体健康结局的局限性。