The design of the query is crucial for the performance of DETR and its variants. Each query consists of two components: a content part and a positional one. Traditionally, the content query is initialized with a zero or learnable embedding, lacking essential content information and resulting in sub-optimal performance. In this paper, we introduce a novel plug-and-play module, Self-Adaptive Content Query (SACQ), to address this limitation. The SACQ module utilizes features from the transformer encoder to generate content queries via self-attention pooling. This allows candidate queries to adapt to the input image, resulting in a more comprehensive content prior and better focus on target objects. However, this improved concentration poses a challenge for the training process that utilizes the Hungarian matching, which selects only a single candidate and suppresses other similar ones. To overcome this, we propose a query aggregation strategy to cooperate with SACQ. It merges similar predicted candidates from different queries, easing the optimization. Our extensive experiments on the COCO dataset demonstrate the effectiveness of our proposed approaches across six different DETR's variants with multiple configurations, achieving an average improvement of over 1.0 AP.

翻译：查询设计对DETR及其变体的性能至关重要。每个查询包含两个组成部分：内容部分和位置部分。传统上，内容查询以零向量或可学习嵌入初始化，缺乏必要的内容信息，导致性能次优。本文提出一种新型即插即用模块——自适应内容查询（SACQ），以解决该局限性。SACQ模块利用Transformer编码器的特征，通过自注意力池化生成内容查询，使候选查询能自适应输入图像，从而获得更全面的内容先验信息，并更聚焦于目标对象。然而，这种增强的聚焦能力给基于匈牙利匹配的训练过程带来挑战——该匹配机制仅选择单一候选查询而抑制其他相似查询。为此，我们提出一种查询聚合策略与SACQ协同工作，该策略合并不同查询生成的相似预测候选结果，从而简化优化过程。在COCO数据集上的大量实验表明，所提方法在六种不同配置的DETR变体上均有效，平均AP提升超过1.0。