Combining model-based and model-free reinforcement learning approaches, this paper proposes and analyzes an $\epsilon$-policy gradient algorithm for the online pricing learning task. The algorithm extends $\epsilon$-greedy algorithm by replacing greedy exploitation with gradient descent step and facilitates learning via model inference. We optimize the regret of the proposed algorithm by quantifying the exploration cost in terms of the exploration probability $\epsilon$ and the exploitation cost in terms of the gradient descent optimization and gradient estimation errors. The algorithm achieves an expected regret of order $\mathcal{O}(\sqrt{T})$ (up to a logarithmic factor) over $T$ trials.

翻译：结合基于模型与无模型的强化学习方法，本文提出并分析了面向在线定价学习任务的$\epsilon$-策略梯度算法。该算法通过用梯度下降步骤替代贪婪策略中的贪心利用操作，扩展了$\epsilon$-贪婪算法，并借助模型推理促进学习过程。我们通过以探索概率$\epsilon$量化探索代价，以梯度下降优化和梯度估计误差量化利用代价，对算法的遗憾值进行了优化。算法在$T$次试验中实现了期望遗憾阶为$\mathcal{O}(\sqrt{T})$（忽略对数因子）。