This paper presents the Relaxed Continuous-Time Actor-critic (RCTAC) algorithm, a method for finding the nearly optimal policy for nonlinear continuous-time (CT) systems with known dynamics and infinite horizon, such as the path-tracking control of vehicles. RCTAC has several advantages over existing adaptive dynamic programming algorithms for CT systems. It does not require the ``admissibility" of the initialized policy or the input-affine nature of controlled systems for convergence. Instead, given any initial policy, RCTAC can converge to an admissible, and subsequently nearly optimal policy for a general nonlinear system with a saturated controller. RCTAC consists of two phases: a warm-up phase and a generalized policy iteration phase. The warm-up phase minimizes the square of the Hamiltonian to achieve admissibility, while the generalized policy iteration phase relaxes the update termination conditions for faster convergence. The convergence and optimality of the algorithm are proven through Lyapunov analysis, and its effectiveness is demonstrated through simulations and real-world path-tracking tasks.

翻译：本文提出松弛连续时间演员-评论家（RCTAC）算法，这是一种为已知动力学和无限时域的非线性连续时间（CT）系统（如车辆路径跟踪控制）寻找近似最优策略的方法。与现有CT系统的自适应动态规划算法相比，RCTAC具有若干优势：它不需要初始策略的“容许性”或受控系统的输入仿射性质即可保证收敛。相反，给定任意初始策略，RCTAC能够收敛至容许策略，并最终为具有饱和控制器的通用非线性系统找到近似最优策略。RCTAC包含两个阶段：预热阶段和广义策略迭代阶段。预热阶段通过最小化哈密顿量的平方来达成容许性，而广义策略迭代阶段通过放宽更新终止条件以实现更快收敛。通过Lyapunov分析证明了算法的收敛性与最优性，并通过仿真和实际路径跟踪任务验证了其有效性。