We introduce FrontierMath, a benchmark of hundreds of original, exceptionally challenging mathematics problems crafted and vetted by expert mathematicians. The questions cover most major branches of modern mathematics -- from computationally intensive problems in number theory and real analysis to abstract questions in algebraic geometry and category theory. Solving a typical problem requires multiple hours of effort from a researcher in the relevant branch of mathematics, and for the upper end questions, multiple days. FrontierMath uses new, unpublished problems and automated verification to reliably evaluate models while minimizing risk of data contamination. Current state-of-the-art AI models solve under 2% of problems, revealing a vast gap between AI capabilities and the prowess of the mathematical community. As AI systems advance toward expert-level mathematical abilities, FrontierMath offers a rigorous testbed that quantifies their progress.

翻译：我们推出了FrontierMath，这是一个包含数百个由专业数学家精心设计并审核的原创性、极具挑战性的数学问题基准。这些问题涵盖了现代数学的主要分支——从数论和实分析中计算密集型问题，到代数几何和范畴论中的抽象问题。解决一个典型问题需要相关数学分支的研究人员投入数小时的努力，而对于高端问题，则需要数天时间。FrontierMath采用全新、未发表的问题和自动化验证机制，在最小化数据污染风险的同时，可靠地评估模型性能。当前最先进的人工智能模型解题率不足2%，揭示了人工智能能力与数学界专业水平之间的巨大差距。随着人工智能系统向专家级数学能力迈进，FrontierMath提供了一个量化其进展的严格测试平台。