Image embeddings are generally assumed to pose limited privacy risk. We challenge this assumption by formalizing semantic leakage as the ability to recover semantic structures from compressed image embeddings. Surprisingly, we show that semantic leakage does not require exact reconstruction of the original image. Preserving local semantic neighborhoods under embedding alignment is sufficient to expose the intrinsic vulnerability of image embeddings. Crucially, this preserved neighborhood structure allows semantic information to propagate through a sequence of lossy mappings. Based on this conjecture, we propose Semantic Leakage from Image Embeddings (SLImE), a lightweight inference framework that reveals semantic information from standalone compressed image embeddings, incorporating a locally trained semantic retriever with off-the-shelf models, without training task-specific decoders. We thoroughly validate each step of the framework empirically, from aligned embeddings to retrieved tags, symbolic representations, and grammatical and coherent descriptions. We evaluate SLImE across a range of open and closed embedding models, including GEMINI, COHERE, NOMIC, and CLIP, and demonstrate consistent recovery of semantic information across diverse inference tasks. Our results reveal a fundamental vulnerability in image embeddings, whereby the preservation of semantic neighborhoods under alignment enables semantic leakage, highlighting challenges for privacy preservation.1

翻译：图像嵌入通常被认为具有有限的隐私风险。我们通过将语义泄露形式化为从压缩图像嵌入中恢复语义结构的能力，对这一假设提出挑战。令人惊讶的是，我们表明语义泄露并不需要原始图像的确切重建。在嵌入对齐下保持局部语义邻域足以暴露图像嵌入的内在脆弱性。至关重要的是，这种保留的邻域结构允许语义信息通过一系列有损映射进行传播。基于这一推测，我们提出了图像嵌入语义泄露（SLImE），这是一个轻量级的推理框架，可从独立的压缩图像嵌入中揭示语义信息。该框架结合了本地训练的语义检索器与现成模型，无需训练特定任务的解码器。我们通过实证彻底验证了框架的每一步，从对齐嵌入到检索到的标签、符号表示，以及语法连贯的描述。我们在包括GEMINI、COHERE、NOMIC和CLIP在内的一系列开放和封闭嵌入模型中评估了SLImE，并证明了在各种推理任务中语义信息的一致恢复。我们的结果揭示了图像嵌入中的一个根本性漏洞，即对齐下语义邻域的保持导致了语义泄露，这突显了隐私保护所面临的挑战。