This paper presents Grammar Reinforcement Learning (GRL), a reinforcement learning algorithm that uses Monte Carlo Tree Search (MCTS) and a transformer architecture that models a Pushdown Automaton (PDA) within a context-free grammar (CFG) framework. Taking as use case the problem of efficiently counting paths and cycles in graphs, a key challenge in network analysis, computer science, biology, and social sciences, GRL discovers new matrix-based formulas for path/cycle counting that improve computational efficiency by factors of two to six w.r.t state-of-the-art approaches. Our contributions include: (i) a framework for generating gramformers that operate within a CFG, (ii) the development of GRL for optimizing formulas within grammatical structures, and (iii) the discovery of novel formulas for graph substructure counting, leading to significant computational improvements.

翻译：本文提出语法强化学习（GRL），这是一种强化学习算法，它结合了蒙特卡洛树搜索（MCTS）以及一个在上下文无关文法（CFG）框架内模拟下推自动机（PDA）的transformer架构。以高效计算图中路径和环（这是网络分析、计算机科学、生物学和社会科学中的一个关键挑战）这一用例为切入点，GRL发现了新的基于矩阵的路径/环计数公式，其计算效率相较于最先进方法提高了二到六倍。我们的贡献包括：（i）一个用于生成在CFG内运行的语法模型（gramformer）的框架，（ii）开发了用于在语法结构内优化公式的GRL算法，以及（iii）发现了用于图子结构计数的新公式，从而带来了显著的计算性能提升。