We propose a reinforcement learning (RL) approach to model optimal exercise strategies for option-type products. We pursue the RL avenue in order to learn the optimal action-value function of the underlying stopping problem. In addition to retrieving the optimal Q-function at any time step, one can also price the contract at inception. We first discuss the standard setting with one exercise right, and later extend this framework to the case of multiple stopping opportunities in the presence of constraints. We propose to approximate the Q-function with a deep neural network, which does not require the specification of basis functions as in the least-squares Monte Carlo framework and is scalable to higher dimensions. We derive a lower bound on the option price obtained from the trained neural network and an upper bound from the dual formulation of the stopping problem, which can also be expressed in terms of the Q-function. Our methodology is illustrated with examples covering the pricing of swing options.

翻译：本文提出一种强化学习方法，用于对期权类产品的最优行权策略进行建模。我们采用强化学习路径来学习基础停止问题的最优动作价值函数。除了获取任意时间步的最优Q函数外，该方法还能在起始时刻对合约定价。我们首先讨论具有单次行权权的标准设定，随后将该框架扩展至存在约束条件下的多重停止机会场景。我们提出使用深度神经网络逼近Q函数，该方法无需像最小二乘蒙特卡洛框架那样设定基函数，且能扩展至高维情形。我们推导了训练神经网络所得期权价格的下界，以及停止问题对偶形式的上界——该上界同样可通过Q函数表示。我们通过包含摆动期权定价的算例对该方法进行了验证。