Recent work shows the effectiveness of Machine Learning (ML) to reduce cache miss ratios by making better eviction decisions than heuristics. However, state-of-the-art ML caches require many predictions to make an eviction decision, making them impractical for high-throughput caching systems. This paper introduces Machine learning At the Tail (MAT), a framework to build efficient ML-based caching systems by integrating an ML module with a traditional cache system based on a heuristic algorithm. MAT treats the heuristic algorithm as a filter to receive high-quality samples to train an ML model and likely candidate objects for evictions. We evaluate MAT on 8 production workloads, spanning storage, in-memory caching, and CDNs. The simulation experiments show MAT reduces the number of costly ML predictions-per-eviction from 63 to 2, while achieving comparable miss ratios to the state-of-the-art ML cache system. We compare a MAT prototype system with an LRU-based caching system in the same setting and show that they achieve similar request rates.

翻译：近期研究表明，机器学习通过做出比启发式算法更优的驱逐决策，能够有效降低缓存缺失率。然而，现有的先进机器学习缓存系统在每次驱逐决策时需要大量预测运算，这使得其难以应用于高吞吐量缓存场景。本文提出"基于机器学习的尾部优化"框架——通过将机器学习模块与传统基于启发式算法的缓存系统相结合，构建高效的机器学习驱动缓存系统。该框架将启发式算法作为过滤器，一方面获取高质量样本用于训练机器学习模型，另一方面筛选出可能的缓存驱逐候选对象。我们在涵盖存储系统、内存缓存和内容分发网络的8个生产环境负载上进行了评估。仿真实验表明，该框架将每次驱逐所需的昂贵机器学习预测次数从63次降至2次，同时实现了与当前先进机器学习缓存系统相近的缺失率。在相同环境下将MAT原型系统与基于LRU的缓存系统进行对比，两者达到了相似的请求处理速率。