Previous Knowledge Distillation based efficient image retrieval methods employs a lightweight network as the student model for fast inference. However, the lightweight student model lacks adequate representation capacity for effective knowledge imitation during the most critical early training period, causing final performance degeneration. To tackle this issue, we propose a Capacity Dynamic Distillation framework, which constructs a student model with editable representation capacity. Specifically, the employed student model is initially a heavy model to fruitfully learn distilled knowledge in the early training epochs, and the student model is gradually compressed during the training. To dynamically adjust the model capacity, our dynamic framework inserts a learnable convolutional layer within each residual block in the student model as the channel importance indicator. The indicator is optimized simultaneously by the image retrieval loss and the compression loss, and a retrieval-guided gradient resetting mechanism is proposed to release the gradient conflict. Extensive experiments show that our method has superior inference speed and accuracy, e.g., on the VeRi-776 dataset, given the ResNet101 as a teacher, our method saves 67.13% model parameters and 65.67% FLOPs (around 24.13% and 21.94% higher than state-of-the-arts) without sacrificing accuracy (around 2.11% mAP higher than state-of-the-arts).

翻译：先前基于知识蒸馏的高效图像检索方法采用轻量级网络作为学生模型以实现快速推理。然而，轻量级学生模型在最为关键的训练初期缺乏足够的表征能力来有效模仿知识，导致最终性能退化。为解决这一问题，我们提出了一种能力动态蒸馏框架，该框架构建了具有可编辑表征能力的学生模型。具体而言，所采用的学生模型最初为重型模型，以便在训练早期充分学习蒸馏知识，随后在训练过程中逐步压缩。为动态调整模型能力，我们的动态框架在学生模型的每个残差块中插入一个可学习卷积层作为通道重要性指示器。该指示器通过图像检索损失和压缩损失联合优化，并提出了检索引导的梯度重置机制以缓解梯度冲突。大量实验表明，我们的方法在推理速度和精度上均具有优越性——例如，在VeRi-776数据集上，以ResNet101为教师模型时，我们的方法在不牺牲精度（mAP比现有最优方法高约2.11%）的情况下，节省了67.13%的模型参数和65.67%的FLOPs（分别比现有最优方法高约24.13%和21.94%）。