This research delves into the role of the quantum Fisher Information Matrix (FIM) in enhancing the performance of Parameterized Quantum Circuit (PQC)-based reinforcement learning agents. While previous studies have highlighted the effectiveness of PQC-based policies preconditioned with the quantum FIM in contextual bandits, its impact in broader reinforcement learning contexts, such as Markov Decision Processes, is less clear. Through a detailed analysis of L\"owner inequalities between quantum and classical FIMs, this study uncovers the nuanced distinctions and implications of using each type of FIM. Our results indicate that a PQC-based agent using the quantum FIM without additional insights typically incurs a larger approximation error and does not guarantee improved performance compared to the classical FIM. Empirical evaluations in classic control benchmarks suggest even though quantum FIM preconditioning outperforms standard gradient ascent, in general it is not superior to classical FIM preconditioning.
翻译:本研究深入探讨了量子Fisher信息矩阵(FIM)在提升基于参数化量子电路(PQC)的强化学习代理性能中的作用。尽管先前的研究已强调,在上下文老虎机问题中,基于PQC的策略经量子FIM预条件后可取得良好效果,但其在更广泛的强化学习场景(如马尔可夫决策过程)中的影响尚不明确。通过对量子与经典FIM的Löwner不等式进行详细分析,本研究揭示了两种FIM之间的微妙差异及其应用含义。结果表明,若缺乏额外洞见,使用量子FIM的PQC代理通常会引入更大的近似误差,且无法保证性能优于经典FIM。在经典控制基准任务中的实证评估表明,尽管量子FIM预条件优于标准梯度上升法,但总体上并未比经典FIM预条件更具优势。