The explosive growth of AI-generated images has created a sustainability challenge for storage infrastructure. Platforms like Midjourney and Adobe Firefly already host billions of generative images, yet conventional object stores persist them as blobs with full-resolution pixels, consuming huge amounts of storage capacity and bandwidth. Unlike natural photos, however, AI-generated images can be deterministically reconstructed from compact, model-native latent tensors, making persistent image storage fundamentally redundant. This paper presents LatentBox, a latent-first storage system for AI-generated images. LatentBox treats compressed latents as durable storage objects and uses on-demand GPU reconstruction on the read path to trade inexpensive compute for large persistent storage savings. Our design is guided by the first large-scale analysis of AI-generated image access we are aware of, based on a 35-month, 2-billion-request production trace from a major generative-content platform. Motivated by the trace analysis, LatentBox keeps frequently accessed images in decoded pixel format for fast hits, stores less-active objects as compressed latents to expand effective cache capacity, and continuously adjusts the splits between the image and latent cache to optimize user-perceived access latency.We build a LatentBox prototype and evaluate it with the production trace. LatentBox reduces persistent storage by 78.7% with competitive or even lower mean and tail latency over a pure image-based storage.
翻译:AI生成图像的爆发式增长给存储基础设施带来了可持续性挑战。Midjourney和Adobe Firefly等平台已托管数十亿张生成图像,传统对象存储将它们以全分辨率像素的二进制大对象持久化,消耗大量存储容量和带宽。然而,与自然照片不同,AI生成图像可从紧凑的模型原生潜在张量中确定性重构,这使得持久图像存储本质上是冗余的。本文提出LatBox——面向AI生成图像的潜在优先存储系统。LatBox将压缩潜在向量视为持久存储对象,并在读取路径上使用按需GPU重构,以廉价计算换取大量持久存储节省。我们的设计基于已知首个大规模AI生成图像访问分析,该分析基于某主要生成内容平台35个月、20亿请求的生产轨迹。受轨迹分析启发,LatBox将高频访问图像以解码像素格式保留以实现快速命中,将低频活跃对象存储为压缩潜在向量以扩展有效缓存容量,并持续调整图像缓存与潜在缓存间的分配以优化用户感知的访问延迟。我们构建了LatBox原型,并使用生产轨迹进行评估。与纯图像存储方案相比,LatBox减少78.7%的持久存储,同时实现更具竞争力甚至更低的平均延迟和尾部延迟。