Container virtualization enables emerging AI workloads such as model serving, highly parallelized training, machine learning pipelines, and so on, to be easily scaled on demand on the elastic cloud infrastructure. Particularly, AI workloads require persistent storage to store data such as training inputs, models, and checkpoints. An external storage system like cloud object storage is a common choice because of its elasticity and scalability. To mitigate access latency to external storage, caching at a local filesystem is an essential technique. However, building local caches on scaling clusters must cope with explosive disk usage, redundant networking, and unexpected failures. We propose objcache, an elastic filesystem over external storage. Objcache introduces an internal transaction protocol over Raft logging to enable atomic updates of distributed persistent states with consistent hashing. The proposed transaction protocol can also manage inode dirtiness by maintaining the consistency between the local cache and external storage. Objcache supports scaling down to zero by automatically evicting dirty files to external storage. Our evaluation reports that objcache speeded up model serving startup by 98.9% compared to direct copies via S3 interfaces. Scaling up with dirty files completed from 2 to 14 seconds with 1024 dirty files.

翻译：容器虚拟化技术支持新兴的人工智能工作负载（如模型服务、高度并行化训练、机器学习流水线等）在弹性云基础设施上按需轻松扩展。特别是，AI工作负载需要持久化存储来保存训练数据、模型和检查点等数据。外部存储系统（如云对象存储）因其弹性和可扩展性成为常见选择。为缓解外部存储的访问延迟，在本地文件系统上进行缓存是一项关键技术。然而，在扩展集群上构建本地缓存必须应对磁盘空间爆炸性增长、网络冗余和意外故障等问题。我们提出Objcache——一种基于外部存储的弹性文件系统。Objcache引入了一种基于Raft日志的内部事务协议，通过一致性哈希实现对分布式持久化状态的原子更新。该事务协议还能通过维护本地缓存与外部存储间的一致性来管理inode脏状态。通过自动将脏文件逐出至外部存储，Objcache支持缩容至零。评估结果显示，与通过S3接口直接复制相比，Objcache将模型服务启动速度提升了98.9%。在包含1024个脏文件时，带脏文件的扩容操作可在2至14秒内完成。