Understanding the transition events between metastable states in complex systems is an important subject in the fields of computational physics, chemistry and biology. The transition pathway plays an important role in characterizing the mechanism underlying the transition, for example, in the study of conformational changes of bio-molecules. In fact, computing the transition pathway is a challenging task for complex and high-dimensional systems. In this work, we formulate the path-finding task as a cost minimization problem over a particular path space. The cost function is adapted from the Freidlin-Wentzell action functional so that it is able to deal with rough potential landscapes. The path-finding problem is then solved using a actor-critic method based on the deep deterministic policy gradient algorithm (DDPG). The method incorporates the potential force of the system in the policy for generating episodes and combines physical properties of the system with the learning process for molecular systems. The exploitation and exploration nature of reinforcement learning enables the method to efficiently sample the transition events and compute the globally optimal transition pathway. We illustrate the effectiveness of the proposed method using three benchmark systems including an extended Mueller system and the Lennard-Jones system of seven particles.

翻译：理解复杂系统中亚稳态之间的过渡事件是计算物理、化学及生物学领域的重要课题。过渡路径在表征过渡机制（例如生物分子构象变化研究）中发挥着关键作用。然而，对于复杂高维系统而言，计算过渡路径是一项极具挑战性的任务。本文将路径搜索问题表述为特定路径空间上的成本最小化问题。通过改写Freidlin-Wentzell作用泛函以兼容粗糙势能面，构建了适应复杂能量地形的成本函数。采用基于深度确定性策略梯度算法（DDPG）的actor-critic方法求解该路径搜索问题：该方法在策略网络中整合系统势能力以生成轨迹片段，并将分子系统的物理特性与学习过程有机结合。强化学习固有的利用-探索机制使该方法能够高效采样过渡事件并计算全局最优过渡路径。通过包括扩展Mueller系统与七粒子Lennard-Jones系统在内的三个基准算例验证了所提方法的有效性。