The battery energy storage system (BESS) has immense potential for enhancing grid reliability and security through its participation in the electricity market. BESS often seeks various revenue streams by taking part in multiple markets to unlock its full potential, but effective algorithms for joint-market participation under price uncertainties are insufficiently explored in the existing research. To bridge this gap, we develop a novel BESS joint bidding strategy that utilizes deep reinforcement learning (DRL) to bid in the spot and contingency frequency control ancillary services (FCAS) markets. Our approach leverages a transformer-based temporal feature extractor to effectively respond to price fluctuations in seven markets simultaneously and helps DRL learn the best BESS bidding strategy in joint-market participation. Additionally, unlike conventional "black-box" DRL model, our approach is more interpretable and provides valuable insights into the temporal bidding behavior of BESS in the dynamic electricity market. We validate our method using realistic market prices from the Australian National Electricity Market. The results show that our strategy outperforms benchmarks, including both optimization-based and other DRL-based strategies, by substantial margins. Our findings further suggest that effective temporal-aware bidding can significantly increase profits in the spot and contingency FCAS markets compared to individual market participation.

翻译：电池储能系统（BESS）通过参与电力市场，在提升电网可靠性与安全性方面具有巨大潜力。为充分发挥其潜能，BESS通常需要参与多个市场以获取多元收益，但现有研究对价格不确定性条件下的联合市场参与算法探索尚不充分。为填补这一研究空白，我们提出了一种新颖的BESS联合投标策略，该策略利用深度强化学习（DRL）在现货与紧急频率控制辅助服务（FCAS）市场中进行投标。所提方法采用基于Transformer的时序特征提取器，能够有效应对七个市场的价格波动，并辅助DRL学习联合市场场景中最优的BESS投标策略。此外，与传统"黑箱"DRL模型不同，本方法具有更强的可解释性，可深入揭示动态电力市场中BESS的时序投标行为规律。我们采用澳大利亚国家电力市场的真实市场价格进行验证。结果表明，相较于基于优化的策略及其他DRL策略等基准方法，我们的策略在收益上实现显著提升。研究进一步表明，与单独市场参与相比，有效的时序感知投标策略可大幅增加现货与紧急FCAS市场的利润。