Various works have utilized deep reinforcement learning (DRL) to address the query optimization problem in database system. They either learn to construct plans from scratch in a bottom-up manner or guide the plan generation behavior of traditional optimizer using hints. While these methods have achieved some success, they face challenges in either low training efficiency or limited plan search space. To address these challenges, we introduce FOSS, a novel DRL-based framework for query optimization. FOSS initiates optimization from the original plan generated by a traditional optimizer and incrementally refines suboptimal nodes of the plan through a sequence of actions. Additionally, we devise an asymmetric advantage model to evaluate the advantage between two plans. We integrate it with a traditional optimizer to form a simulated environment. Leveraging this simulated environment, FOSS can bootstrap itself to rapidly generate a large amount of high-quality simulated experiences. FOSS then learns and improves its optimization capability from these simulated experiences. We evaluate the performance of FOSS on Join Order Benchmark, TPC-DS, and Stack Overflow. The experimental results demonstrate that FOSS outperforms the state-of-the-art methods in terms of latency performance and optimization time. Compared to PostgreSQL, FOSS achieves savings ranging from 15% to 83% in total latency across different benchmarks.

翻译：多项工作已利用深度强化学习（DRL）解决数据库系统中的查询优化问题。这些方法要么采用自底向上的方式从头学习构建执行计划，要么通过提示信息引导传统优化器的计划生成行为。尽管这些方法取得了一定成功，但仍面临训练效率低下或计划搜索空间受限等挑战。为应对这些挑战，我们提出FOSS——一种基于DRL的新型查询优化框架。FOSS从传统优化器生成的原始计划出发，通过一系列动作逐步优化计划中的次优节点。此外，我们设计了一种非对称优势模型来评估两个计划间的优劣，并将其与传统优化器集成形成仿真环境。借助该仿真环境，FOSS能够自举式快速生成大量高质量仿真经验，并从中学习提升优化能力。我们在Join Order Benchmark、TPC-DS和Stack Overflow数据集上评估了FOSS的性能。实验结果表明，在延迟性能和优化时间方面，FOSS均优于现有最先进方法。与PostgreSQL相比，FOSS在不同基准测试中的总延迟节省幅度达15%至83%。