This paper presents an approach, Spectral Dynamics Embedding Control (SDEC), to optimal control for nonlinear stochastic systems. This method leverages an infinite-dimensional feature to linearly represent the state-action value function and exploits finite-dimensional truncation approximation for practical implementation. To characterize the effectiveness of these finite dimensional approximations, we provide an in-depth theoretical analysis to characterize the approximation error induced by the finite-dimension truncation and statistical error induced by finite-sample approximation in both policy evaluation and policy optimization. Our analysis includes two prominent kernel approximation methods: truncations onto random features and Nystrom features. We also empirically test the algorithm and compare the performance with Koopman-based, iLQR, and energy-based methods on a few benchmark problems.

翻译：本文提出了一种名为谱动态嵌入控制（SDEC）的方法，用于非线性随机系统的最优控制。该方法利用无限维特征线性表示状态-动作价值函数，并通过有限维截断近似实现实际应用。为刻画这些有限维近似的有效性，我们提供了深入的理论分析，分别评估了策略评估与策略优化中由有限维截断引起的近似误差以及由有限样本近似引起的统计误差。分析涵盖两种主流核函数近似方法：随机特征截断与Nyström特征截断。此外，我们在多个基准问题上进行了算法实证测试，并将性能与基于Koopman、iLQR及能量方法进行了对比。