This paper proposes an introspective deep metric learning (IDML) framework for uncertainty-aware comparisons of images. Conventional deep metric learning methods produce confident semantic distances between images regardless of the uncertainty level. However, we argue that a good similarity model should consider the semantic discrepancies with caution to better deal with ambiguous images for more robust training. To achieve this, we propose to represent an image using not only a semantic embedding but also an accompanying uncertainty embedding, which describes the semantic characteristics and ambiguity of an image, respectively. We further propose an introspective similarity metric to make similarity judgments between images considering both their semantic differences and ambiguities. The proposed IDML framework improves the performance of deep metric learning through uncertainty modeling and attains state-of-the-art results on the widely used CUB-200-2011, Cars196, and Stanford Online Products datasets for image retrieval and clustering. We further provide an in-depth analysis of our framework to demonstrate the effectiveness and reliability of IDML. Code is available at: https://github.com/wzzheng/IDML.

翻译：本文提出了一种内省式深度度量学习框架，用于实现不确定性感知的图像比较。传统的深度度量学习方法无论不确定性水平如何，都会生成置信的语义距离。然而，我们认为一个良好的相似性模型应谨慎考虑语义差异，以更好地处理模糊图像，从而实现更稳健的训练。为此，我们提出不仅使用语义嵌入表示图像，还引入相应的不确定性嵌入，分别描述图像的语义特征与模糊性。进一步地，我们提出一种内省式相似性度量，在考虑语义差异和不确定性的前提下进行图像间相似性判断。所提出的IDML框架通过不确定性建模提升了深度度量学习的性能，并在广泛使用的CUB-200-2011、Cars196和Stanford Online Products数据集上，针对图像检索和聚类任务取得了最先进的结果。此外，我们对框架进行了深入分析，以证明IDML的有效性和可靠性。代码开源地址：https://github.com/wzzheng/IDML。