Hybrid storage systems (HSS) use multiple different storage devices to provide high and scalable storage capacity at high performance. Recent research proposes various techniques that aim to accurately identify performance-critical data to place it in a "best-fit" storage device. Unfortunately, most of these techniques are rigid, which (1) limits their adaptivity to perform well for a wide range of workloads and storage device configurations, and (2) makes it difficult for designers to extend these techniques to different storage system configurations (e.g., with a different number or different types of storage devices) than the configuration they are designed for. We introduce Sibyl, the first technique that uses reinforcement learning for data placement in hybrid storage systems. Sibyl observes different features of the running workload as well as the storage devices to make system-aware data placement decisions. For every decision it makes, Sibyl receives a reward from the system that it uses to evaluate the long-term performance impact of its decision and continuously optimizes its data placement policy online. We implement Sibyl on real systems with various HSS configurations. Our results show that Sibyl provides 21.6%/19.9% performance improvement in a performance-oriented/cost-oriented HSS configuration compared to the best previous data placement technique. Our evaluation using an HSS configuration with three different storage devices shows that Sibyl outperforms the state-of-the-art data placement policy by 23.9%-48.2%, while significantly reducing the system architect's burden in designing a data placement mechanism that can simultaneously incorporate three storage devices. We show that Sibyl achieves 80% of the performance of an oracle policy that has complete knowledge of future access patterns while incurring a very modest storage overhead of only 124.4 KiB.

翻译：混合存储系统（HSS）利用多种不同存储设备，在提供高性能的同时实现高可扩展存储容量。近年来研究提出了多种技术，旨在准确识别性能关键数据并将其放置到"最佳适配"的存储设备中。然而，这些技术大多存在刚性问题：（1）其工作负载和存储设备配置的适应性受限，难以在广泛场景中保持良好性能；（2）设计者难以将这些技术扩展到与原始设计配置不同的存储系统配置（如不同数量或类型的存储设备）。我们提出Sibyl，这是首个在混合存储系统中利用强化学习进行数据放置的技术。Sibyl通过观测运行工作负载及存储设备的多维特征，做出系统感知的数据放置决策。对于每次决策，Sibyl从系统中获取奖励信号，据此评估该决策的长期性能影响，并在线持续优化其数据放置策略。我们在不同HSS配置的真实系统上实现了Sibyl。实验结果表明，相较于先前最优的数据放置技术，Sibyl在面向性能/面向成本的HSS配置中分别实现了21.6%/19.9%的性能提升。在包含三种不同存储设备的HSS配置评估中，Sibyl相比当前最优数据放置策略性能提升23.9%-48.2%，同时显著降低了系统架构师设计可同时容纳三种存储设备的数据放置机制的工作负担。我们证明Sibyl能达到具备完全未来访问模式知识的神谕策略80%的性能，而存储开销仅为124.4 KiB。