Deep or reinforcement learning (RL) approaches have been adapted as reactive agents to quickly learn and respond with new investment strategies for portfolio management under the highly turbulent financial market environments in recent years. In many cases, due to the very complex correlations among various financial sectors, and the fluctuating trends in different financial markets, a deep or reinforcement learning based agent can be biased in maximising the total returns of the newly formulated investment portfolio while neglecting its potential risks under the turmoil of various market conditions in the global or regional sectors. Accordingly, a multi-agent and self-adaptive framework namely the MASA is proposed in which a sophisticated multi-agent reinforcement learning (RL) approach is adopted through two cooperating and reactive agents to carefully and dynamically balance the trade-off between the overall portfolio returns and their potential risks. Besides, a very flexible and proactive agent as the market observer is integrated into the MASA framework to provide some additional information on the estimated market trends as valuable feedbacks for multi-agent RL approach to quickly adapt to the ever-changing market conditions. The obtained empirical results clearly reveal the potential strengths of our proposed MASA framework based on the multi-agent RL approach against many well-known RL-based approaches on the challenging data sets of the CSI 300, Dow Jones Industrial Average and S&P 500 indexes over the past 10 years. More importantly, our proposed MASA framework shed lights on many possible directions for future investigation.

翻译：摘要：近年来，深度或强化学习方法已被改造为反应式智能体，能够在高度动荡的金融市场环境中快速学习并制定新的投资组合管理策略。然而，由于各金融板块之间极为复杂的关联性以及不同金融市场的波动趋势，基于深度或强化学习的智能体往往倾向于最大化新构建投资组合的总收益，却可能忽视在全球或区域市场动荡条件下的潜在风险。为此，本文提出了一种名为MASA的多智能体自适应框架，该框架采用复杂的多智能体强化学习方法，通过两个相互协作的反应式智能体，在整体投资组合收益与潜在风险之间进行谨慎而动态的平衡。此外，MASA框架还集成了一个高度灵活且具有主动性的市场观察者智能体，为多智能体强化学习方法提供关于预估市场趋势的额外信息，作为有价值的反馈，使其能够快速适应不断变化的市场环境。基于沪深300、道琼斯工业平均指数和标普500指数过去10年的挑战性数据集，实证结果清晰地揭示了我们所提出的基于多智能体强化学习的MASA框架相对于众多知名强化学习方法所具备的潜在优势。更重要的是，本文提出的MASA框架为未来的研究探索指明了多个可能的方向。