A common architectural choice for deep metric learning is a convolutional neural network followed by global average pooling (GAP). Albeit simple, GAP is a highly effective way to aggregate information. One possible explanation for the effectiveness of GAP is considering each feature vector as representing a different semantic entity and GAP as a convex combination of them. Following this perspective, we generalize GAP and propose a learnable generalized sum pooling method (GSP). GSP improves GAP with two distinct abilities: i) the ability to choose a subset of semantic entities, effectively learning to ignore nuisance information, and ii) learning the weights corresponding to the importance of each entity. Formally, we propose an entropy-smoothed optimal transport problem and show that it is a strict generalization of GAP, i.e., a specific realization of the problem gives back GAP. We show that this optimization problem enjoys analytical gradients enabling us to use it as a direct learnable replacement for GAP. We further propose a zero-shot loss to ease the learning of GSP. We show the effectiveness of our method with extensive evaluations on 4 popular metric learning benchmarks. Code is available at: GSP-DML Framework

翻译：深度度量学习中常见的架构选择是卷积神经网络后接全局平均池化（GAP）。尽管结构简单，GAP是一种高效的信息聚合方法。对其有效性的一种可能解释是将每个特征向量视为不同语义实体的表征，而GAP则是对这些实体的凸组合。基于这一视角，我们对GAP进行泛化，并提出一种可学习的广义求和池化方法（GSP）。GSP通过两种独特能力改进了GAP：i) 能够选择语义实体的子集，有效学习忽略干扰信息；ii) 学习与每个实体重要性相对应的权重。形式上，我们提出一个熵平滑最优传输问题，并证明该问题是GAP的严格泛化——该问题的特定实例化可还原为GAP。我们证明该优化问题具有解析梯度，使其可作为GAP的直接可学习替代方案。此外我们提出零样本损失以简化GSP的学习。通过在4个主流度量学习基准上的广泛评估，验证了方法的有效性。代码开源地址：GSP-DML Framework