Artificial intelligence-based systems for player risk detection have become central to harm prevention efforts in the gambling industry. However, growing concerns around transparency and effectiveness have highlighted the absence of standardized methods for evaluating the quality and impact of these tools. This makes it impossible to gauge true progress; even as new systems are developed, their comparative effectiveness remains unknown. We argue the critical next innovation is developing a framework to measure these systems. This paper proposes a conceptual benchmarking framework to support the systematic evaluation of player risk detection systems. Benchmarking, in this context, refers to the structured and repeatable assessment of artificial intelligence models using standardized datasets, clearly defined tasks, and agreed-upon performance metrics. The goal is to enable objective, comparable, and longitudinal evaluation of player risk detection systems. We present a domain-specific framework for benchmarking that addresses the unique challenges of player risk detection in gambling and supports key stakeholders, including researchers, operators, vendors, and regulators. By enhancing transparency and improving system effectiveness, this framework aims to advance innovation and promote responsible artificial intelligence adoption in gambling harm prevention.

翻译：基于人工智能的玩家风险检测系统已成为赌博行业危害预防工作的核心。然而，围绕透明度和有效性的日益增长的关切凸显了评估这些工具质量和影响的标准化方法的缺失。这使得衡量真实进展变得不可能；即使新系统被开发出来，其相对有效性仍属未知。我们认为，下一步关键的创新在于建立一个衡量这些系统的框架。本文提出了一个概念性基准测试框架，以支持对玩家风险检测系统的系统性评估。在此语境下，基准测试指的是使用标准化数据集、明确定义的任务和公认的性能指标，对人工智能模型进行结构化且可重复的评估。其目标是实现对玩家风险检测系统的客观、可比较和纵向评估。我们提出了一个针对特定领域的基准测试框架，该框架应对赌博中玩家风险检测的独特挑战，并支持包括研究人员、运营商、供应商和监管机构在内的关键利益相关者。通过增强透明度和提高系统有效性，该框架旨在推动创新，并促进负责任的人工智能在赌博危害预防领域的应用。