This work seeks to center validity considerations in deliberations around whether and how to build data-driven algorithms in high-stakes domains. Toward this end, we translate key concepts from validity theory to predictive algorithms. We describe common challenges in problem formulation and data issues that jeopardize the validity of predictive algorithms. We distill these issues into a series of high-level questions intended to promote and document reflections on the legitimacy of the predictive task and the suitability of the data. This contribution lays the foundation for co-designing a validity protocol, in collaboration with real-world stakeholders, including decision-makers, modelers, and members of potentially impacted communities, to critically evaluate the justifiability of specific designs and uses of data-driven algorithmic systems.

翻译：这项工作力求在围绕是否和如何在高取量领域建立数据驱动算法的审议中集中考虑有效性问题。为此,我们将关键概念从有效性理论转变为预测性算法。我们描述了在问题拟订和数据问题上危及预测性算法有效性的共同挑战。我们将这些问题纳入一系列高级别问题,目的是促进和记录对预测性任务的合法性和数据是否合适的思考。这一贡献为与现实世界利益攸关方,包括决策者、建模者和潜在受影响社区成员合作共同设计有效性协议奠定了基础,以便严格评估数据驱动算法系统的具体设计和使用是否合理。