Deep reinforcement learning (DRL) has revolutionized quantitative finance by achieving excellent performance without significant manual effort. Whereas we observe that the DRL models behave unstably in a dynamic stock market due to the low signal-to-noise ratio nature of the financial data. In this paper, we propose a novel logic-guided trading framework, termed as SYENS (Program Synthesis-based Ensemble Strategy). Different from the previous state-of-the-art ensemble reinforcement learning strategy which arbitrarily selects the best-performing agent for testing based on a single measurement, our framework proposes regularizing the model's behavior in a hierarchical manner using the program synthesis by sketching paradigm. First, we propose a high-level, domain-specific language (DSL) that is used for the depiction of the market environment and action. Then based on the DSL, a novel program sketch is introduced, which embeds human expert knowledge in a logical manner. Finally, based on the program sketch, we adopt the program synthesis by sketching a paradigm and synthesizing a logical, hierarchical trading strategy. We evaluate SYENS on the 30 Dow Jones stocks under the cash trading and the margin trading settings. Experimental results demonstrate that our proposed framework can significantly outperform the baselines with much higher cumulative return and lower maximum drawdown under both settings.

翻译：深度强化学习（DRL）通过无需大量人工干预即可实现卓越性能，彻底改变了量化金融领域。然而，我们观察到，由于金融数据的低信噪比特性，DRL模型在动态股票市场中表现不稳定。在本文中，我们提出了一种新颖的逻辑引导交易框架，称为SYENS（基于程序合成的集成策略）。与之前最先进的集成强化学习策略（该策略基于单一度量任意选择表现最佳的代理进行测试）不同，我们的框架提出利用草图编程范式，以分层方式正则化模型行为。首先，我们设计了一种高层、领域特定语言（DSL），用于描述市场环境和动作。随后，基于该DSL，引入了一种新颖的程序草图，以逻辑方式嵌入人类专家知识。最后，基于该程序草图，我们采用草图编程范式，合成一种逻辑化、分层的交易策略。我们在现金交易和保证金交易设置下，对道琼斯30只股票评估了SYENS。实验结果表明，我们的框架在两种设置下均能显著超越基线方法，实现更高的累积收益率和更低的最大回撤。