In this article I propose an approach for defining replicability for prediction rules. Motivated by a recent NAS report, I start from the perspective that replicability is obtaining consistent results across studies suitable to address the same prediction question, each of which has obtained its own data. I then discuss concept and issues in defining key elements of this statement. I focus specifically on the meaning of "consistent results" in typical utilization contexts, and propose a multi-agent framework for defining replicability, in which agents are neither partners nor adversaries. I recover some of the prevalent practical approaches as special cases. I hope to provide guidance for a more systematic assessment of replicability in machine learning.

翻译：本文提出了一种定义预测规则可复现性的方法。受美国国家科学院近期报告启发，我从以下视角出发：可复现性是指针对同一预测问题的不同研究（各自独立获取数据）获得一致结果。随后，我探讨了界定这一表述关键要素时的概念与问题，特别聚焦于典型应用场景中“一致结果”的含义，并提出一个多智能体框架来定义可复现性——其中智能体既非合作伙伴亦非对抗者。我将部分现行实用方法作为该框架的特例进行了归纳。希望本研究能为机器学习中可复现性的系统性评估提供指导。