Owning to the unremitting efforts by a few institutes, significant progress has recently been made in designing superhuman AIs in No-limit Texas Hold'em (NLTH), the primary testbed for large-scale imperfect-information game research. However, it remains challenging for new researchers to study this problem since there are no standard benchmarks for comparing with existing methods, which seriously hinders further developments in this research area. In this work, we present OpenHoldem, an integrated toolkit for large-scale imperfect-information game research using NLTH. OpenHoldem makes three main contributions to this research direction: 1) a standardized evaluation protocol for thoroughly evaluating different NLTH AIs, 2) four publicly available strong baselines for NLTH AI, and 3) an online testing platform with easy-to-use APIs for public NLTH AI evaluation. We have released OpenHoldem at holdem.ia.ac.cn, hoping it facilitates further studies on the unsolved theoretical and computational issues in this area and cultivate crucial research problems like opponent modeling and human-computer interactive learning.

翻译：由于少数几个研究所作出了不懈的努力,最近在设计“不限量的得克萨斯控股公司(NLTH)”这一大规模不完善信息游戏研究的主要测试台的超人AI(NLTH)方面取得了显著进展,然而,由于没有标准基准来比较现有方法,严重妨碍这一研究领域的进一步发展,因此新的研究人员仍难以研究这一问题。在这项工作中,我们介绍了利用NLTH进行大规模不完善信息游戏研究的综合工具包OpenHoldem。 OpenHoldem对这一研究方向作出了三大主要贡献:1) 全面评估不同NLTH AI的标准化评价协议,2) NLTH AI 4 4 可供公开使用的强大基线,3) 以及3) 公开使用方便使用API的在线测试平台,用于公众NLTH AI 评估。我们发布了“Holdem ” 的“ oldem at holdem sholdem at holdem.ia. a. a. a. a. c. c.n”,希望它能促进对该领域的未解决的理论和计算问题进行进一步研究,并产生关键的研究问题。