Content Delivery Networks carry the majority of Internet traffic, and the increasing demand for video content as a major IP traffic across the Internet highlights the importance of caching and prefetching optimization algorithms. Prefetching aims to make data available in the cache before the requester places its request to reduce access time and improve the Quality of Experience on the user side. Prefetching is well investigated in operating systems, compiler instructions, in-memory cache, local storage systems, high-speed networks, and cloud systems. Traditional prefetching techniques are well adapted to a particular access pattern, but fail to adapt to sudden variations or randomization in workloads. This paper explores the use of reinforcement learning to tackle the changes in user access patterns and automatically adapt over time. To this end, we propose, DeePref, a Deep Reinforcement Learning agent for online video content prefetching in Content Delivery Networks. DeePref is a prefetcher implemented on edge networks and is agnostic to hardware design, operating systems, and applications. Our results show that DeePref DRQN, using a real-world dataset, achieves a 17% increase in prefetching accuracy and a 28% increase in prefetching coverage on average compared to baseline approaches that use video content popularity as a building block to statically or dynamically make prefetching decisions. We also study the possibility of transfer learning of statistical models from one edge network into another, where unseen user requests from unknown distribution are observed. In terms of transfer learning, the increase in prefetching accuracy and prefetching coverage are [$30%$, $10%$], respectively. Our source code will be available on Github.

翻译：内容分发网络承载了大部分的互联网流量，而视频内容作为互联网上主要的IP流量，其需求的日益增长凸显了缓存与预取优化算法的重要性。预取旨在请求者发起请求前将数据提前存入缓存，以降低访问时延、提升用户体验质量。预取技术已在操作系统、编译器指令、内存缓存、本地存储系统、高速网络及云系统中得到广泛研究。传统预取技术适用于特定访问模式，但难以适应工作负载的突发变化或随机化。本文探索利用强化学习应对用户访问模式的变化，并实现随时间自动自适应。为此，我们提出了DeePref——一种用于内容分发网络在线视频内容预取的深度强化学习智能体。DeePref是部署在边缘网络上的预取器，且与硬件设计、操作系统及应用程序无关。实验结果表明，基于真实数据集的DeePref DRQN在预取准确率上平均提升17%，在预取覆盖率上平均提升28%，优于以视频内容流行度为基础进行静态或动态预取决策的基准方法。我们还研究了统计模型从边缘网络向另一个边缘网络的迁移学习可能性，其中需要处理来自未知分布且之前未见过的用户请求。在迁移学习方面，预取准确率和预取覆盖率分别提升了[$30%$, $10%$]。我们的源代码将发布在Github上。