The explosive growth of AI-generated images has created a sustainability challenge for storage infrastructure. Platforms like Midjourney and Adobe Firefly already host billions of generative images, yet conventional object stores persist them as blobs with full-resolution pixels, consuming huge amounts of storage capacity and bandwidth. Unlike natural photos, however, AI-generated images can be deterministically reconstructed from compact, model-native latent tensors, making persistent image storage fundamentally redundant. This paper presents LatentBox, a latent-first storage system for AI-generated images. LatentBox treats compressed latents as durable storage objects and uses on-demand GPU reconstruction on the read path to trade inexpensive compute for large persistent storage savings. Our design is guided by the first large-scale analysis of AI-generated image access we are aware of, based on a 35-month, 2-billion-request production trace from a major generative-content platform. Motivated by the trace analysis, LatentBox keeps frequently accessed images in decoded pixel format for fast hits, stores less-active objects as compressed latents to expand effective cache capacity, and continuously adjusts the splits between the image and latent cache to optimize user-perceived access latency.We build a LatentBox prototype and evaluate it with the production trace. LatentBox reduces persistent storage by 78.7% with competitive or even lower mean and tail latency over a pure image-based storage.
翻译:AI生成图像的爆炸式增长给存储基础设施带来了可持续性挑战。Midjourney和Adobe Firefly等平台已托管数十亿张生成图像,但传统对象存储将其以全分辨率像素形式作为数据块持久化,消耗大量存储容量和带宽。然而,与自然照片不同,AI生成图像可从紧凑的、模型原生的潜张量中确定性重建,使得持久化图像存储本质上冗余。本文提出LatentBox,一种面向AI生成图像的潜优先存储系统。LatentBox将压缩后的潜变量作为持久化存储对象,并在读取路径上利用按需GPU重建,以廉价计算换取大规模持久化存储节省。我们的设计基于已知首次对AI生成图像访问行为的大规模分析,该分析基于某主流生成内容平台35个月、20亿请求的生产环境日志。受日志分析启发,LatentBox将频繁访问的图像保持为解码后的像素格式以实现快速命中,将不活跃对象存储为压缩潜变量以扩展有效缓存容量,并持续调整图像缓存与潜缓存之间的分配比例以优化用户感知的访问延迟。我们构建了LatentBox原型,并利用生产环境日志进行评估。与纯图像存储相比,LatentBox将持久化存储减少78.7%,同时平均延迟和尾延迟具有竞争力甚至更低。