Achieving both optimality and safety under unknown system dynamics is a central challenge in real-world deployment of agents. To address this, we introduce a notion of maximum safe dynamics learning, where sufficient exploration is performed within the space of safe policies. Our method executes $\textit{pessimistically}$ safe policies while $\textit{optimistically}$ exploring informative states and, despite not reaching them due to model uncertainty, ensures continuous online learning of dynamics. The framework achieves first-of-its-kind results: learning the dynamics model sufficiently $-$ up to an arbitrary small tolerance (subject to noise) $-$ in a finite time, while ensuring provably safe operation throughout with high probability and without requiring resets. Building on this, we propose an algorithm to maximize rewards while learning the dynamics $\textit{only to the extent needed}$ to achieve close-to-optimal performance. Unlike typical reinforcement learning (RL) methods, our approach operates online in a non-episodic setting and ensures safety throughout the learning process. We demonstrate the effectiveness of our approach in challenging domains such as autonomous car racing and drone navigation under aerodynamic effects $-$ scenarios where safety is critical and accurate modeling is difficult.

翻译：在未知系统动力学下同时实现最优性与安全性是智能体实际部署中的核心挑战。为此，我们提出了一种最大安全动力学学习的概念，即在安全策略空间内进行充分的探索。我们的方法执行**悲观**安全策略，同时**乐观**探索信息丰富的状态，尽管由于模型不确定性无法到达这些状态，但确保了动力学的持续在线学习。该框架实现了首创性成果：在有限时间内充分学习动力学模型——达到任意小的容忍度（受噪声影响）——同时以高概率保证整个过程中的可证明安全运行，且无需重置。在此基础上，我们提出一种算法，在仅学习**达到接近最优性能所需程度**的动力学的同时最大化奖励。与典型的强化学习方法不同，我们的方法在非分幕式设置中在线运行，并确保整个学习过程的安全性。我们在自动驾驶赛车和空气动力学效应下的无人机导航等具有挑战性的领域验证了方法的有效性——这些场景中安全性至关重要且精确建模困难。