The Method of Successive Approximations (MSA) is a fixed-point iterative method used to solve stochastic optimal control problems. It is an indirect method based on the conditions derived from the Stochastic Maximum Principle (SMP), an extension of the Pontryagin Maximum Principle (PMP) to stochastic control problems. In this study, we investigate the contractivity and the convergence of MSA for a specific and interesting class of stochastic dynamical systems (when the drift coefficient is one-sided-Lipschitz with a negative constant and the diffusion coefficient is Lipschitz continuous). Our analysis unfolds in three key steps: firstly, we prove the stability of the state process with respect to the control process. Secondly, we establish the stability of the adjoint process. Finally, we present rigorous evidence to prove the contractivity and then the convergence of MSA. This study contributes to enhancing the understanding of MSA's applicability and effectiveness in addressing stochastic optimal control problems.
翻译:逐次逼近法(MSA)是一种用于求解随机最优控制问题的定点迭代方法。它是一种基于随机最大值原理(SMP)推导出的条件的间接方法,该原理是庞特里亚金最大值原理(PMP)在随机控制问题中的推广。在本研究中,我们针对一类特定且有趣的随机动力系统(漂移系数为具有负常数的单侧Lipschitz连续,扩散系数为Lipschitz连续),研究了MSA的压缩性与收敛性。我们的分析分三个关键步骤展开:首先,我们证明了状态过程相对于控制过程的稳定性;其次,我们建立了伴随过程的稳定性;最后,我们提供了严格的证据来证明MSA的压缩性,进而证明其收敛性。本研究有助于加深对MSA在解决随机最优控制问题中的适用性和有效性的理解。