Machine learning is traditionally studied at the model level: researchers measure and improve the accuracy, robustness, bias, efficiency, and other dimensions of specific models. In practice, the societal impact of machine learning is determined by the surrounding context of machine learning deployments. To capture this, we introduce ecosystem-level analysis: rather than analyzing a single model, we consider the collection of models that are deployed in a given context. For example, ecosystem-level analysis in hiring recognizes that a job candidate's outcomes are not only determined by a single hiring algorithm or firm but instead by the collective decisions of all the firms they applied to. Across three modalities (text, images, speech) and 11 datasets, we establish a clear trend: deployed machine learning is prone to systemic failure, meaning some users are exclusively misclassified by all models available. Even when individual models improve at the population level over time, we find these improvements rarely reduce the prevalence of systemic failure. Instead, the benefits of these improvements predominantly accrue to individuals who are already correctly classified by other models. In light of these trends, we consider medical imaging for dermatology where the costs of systemic failure are especially high. While traditional analyses reveal racial performance disparities for both models and humans, ecosystem-level analysis reveals new forms of racial disparity in model predictions that do not present in human predictions. These examples demonstrate ecosystem-level analysis has unique strengths for characterizing the societal impact of machine learning.

翻译：机器学习传统上在模型层面进行研究：研究人员测量并提升特定模型的准确率、鲁棒性、偏差、效率及其他维度。在实践中，机器学习的社会影响由其部署的周边环境所决定。为捕捉这一特性，我们引入生态系统级分析：并非分析单个模型，而是考虑在特定情境下部署的所有模型集合。例如，招聘领域的生态系统级分析认识到，求职者的结果不仅由单个招聘算法或企业决定，而是由所有应聘企业的集体决策共同决定。跨越三种模态（文本、图像、语音）及11个数据集，我们确立了一个明确趋势：已部署的机器学习系统易出现系统性故障，即部分用户被所有可用模型错误分类。即使个体模型在种群层面随时间改进，我们发现这些改进很少降低系统性故障的发生率。相反，这些改进的收益主要累积在已被其他模型正确分类的个体身上。基于这些趋势，我们聚焦于皮肤病学领域的医学影像分析——该领域系统性故障的成本尤为高昂。传统分析揭示了模型与人类在种族差异上的表现不均衡，而生态系统级分析则揭示了模型预测中人类预测并不存在的新型种族差异形式。这些案例表明，生态系统级分析在刻画机器学习社会影响方面具有独特优势。